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ANALYZING  THE  EFFECTS 

OF  THE  WEAPON  SYSTEMS  ACQUISITION  REFORM  ACT 


ABSTRACT 


The  purpose  of  this  project  is  to  analyze  the  effects  of  Title  II  Weapon  Systems 
Acquisition  Reform  Act  (WSARA)  of  2009  on  the  Program  Executive  Office  (PEO) 
Combat  Support  and  Combat  Service  Support  (CS&CSS).  Our  methodology  includes  a 
literature  review  as  well  as  interviews  with  program  office  stakeholders.  We  review 
collected  data  to  establish  a  correlation  between  WSARA  implementation  and  how 
stakeholders  perceive  the  application  of  these  regulations  through  the  following; 

•  Identify  the  intended  effect  of  Title  II  WSARA  regulations  on  the 
acquisition  process  concerning  cost,  schedule,  and  performance. 

•  Determine  the  extent  to  which  enacted  Title  II  WSARA  has  been  applied 
within  PEO  CS&CSS  programs. 

•  Identify  gaps  between  the  intent  and  application  of  Title  II  WSARA 
within  PEO  CS&CSS,  offering  recommendations  for  further  research. 

The  analysis  shows  that  correlations  exist  between  WSARA  implementation  and 
improved  cost  and  schedule  performance. 
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I.  INTRODUCTION 


A,  ACQUISITION  PROCESS  AND  THE  PERCEPTIONS  OF  REFORM 

Defense  spending  eurrently  aeeounts  for  over  13%  of  U.S.  government  outlays,  at 
an  antieipated  dollar  amount  of  $800  billion  for  fiseal  year  (FY)  2014.  Aequisitions,  or 
proeurement  (these  terms  are  used  interehangeably),  spending  is  approximately  17.5%  of 
defense  outlays,  or  nearly  $140  billion  annually  (projeeted  for  FY2014).  These  numbers 
give  an  order  of  magnitude  to  the  funds  eommitted  annually  to  ereate  eapabilities  within 
the  U.S.  Department  of  Defense  (Government  Spending  Breakdown,  n.d.). 

Coneern  over  the  aequisition  proeess  appears  warranted  when  eonsidering  the 
amount  of  taxpayer  dollars  funding  this  proeess.  The  Weapon  Systems  Aequisition 
Reform  Aet  (WSARA)  of  2009  is  one  of  many  regulatory  reforms  aimed  at  improving 
the  eomplex  aequisition  proeess.  The  following  seetions  present  a  brief  baekground  on 
the  aequisition  proeess  (system)  and  historieal  efforts  at  reforming  this  proeess  leading  up 
to  the  implementation  of  the  WSARA  in  2009. 

B,  THE  ACQUISITION  SYSTEM  “BIG  A” 

The  aequisition  system  is  eomposed  of  the  requirements  identifieation  proeess, 
the  budgeting  and  funding  proeess,  and  the  aetual  process  of  managing  the  purchase  or 
manufacture  of  the  product.  In  defense  terminology,  these  three  areas  are  as  follows: 

•  The  Joint  Capabilities  Integration  and  Development  System  (JCIDS): 

This  system  is  responsible  for  determining  requirements  and  reducing 
program  redundancy  across  military  service  branches. 

•  The  Planning,  Programming,  Budgeting,  and  Execution  System 
(PPBES):  This  system  is  responsible  for  forecasting  funding  resources 
and  managing  the  execution  of  congressionally  allocated  and  appropriated 
funds. 

•  The  defense  acquisition  system  (DAS):  This  is  the  actual  process  of 
incorporating  user  requirements,  determining  whether  to  make  or  buy  a 
product  to  meet  those  requirements,  and  the  method  spanning  from  this 
requirement  identification  through  the  entire  life  cycle  of  the  resulting 
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product.  This  is  also  the  focal  point  for  acquisition  reform  efforts 
(Schwartz,  2013). 

When  meshed  together,  these  three  areas  contribute  to  the  entire  acquisition 
process  (“Big  A”),  as  depicted  in  Figure  1. 


Figure  1.  The  Acquisition  System  (from  Schwartz,  2013) 


C.  THE  PLANNING,  PROGRAMMING,  BUDGETING,  AND  EXECUTION 
SYSTEM 

“A  number  of  issues  related  to  planning  and  budgeting  for  national  defense  confound 
DOD  and  congressional  decision  makers  annually.  Among  these  are  how  to  perform 
effective  and  competent  threat  assessment  and  the  consequences  of  doing  this  job  well  or 
poorly”  (Jones  &  McCaffery,  2008,  p.  88).  If  there  were  no  need  to  consider  fiscal 
constraints  when  outfitting  U.S.  military  forces,  there  would  be  no  need  for  the  PPBES. 
However,  in  an  environment  of  constraints,  the  PPBES  is  the  U.S.  government’s  method 
of  accounting  for,  planning,  programming,  and  budgeting  for  products  and  systems  to 
meet  the  needs  of  warfighters  and  monitor  the  execution  of  funds  to  ensure  appropriate 
fund  allocation  and  expenditures.  The  PPBES  is  composed  of  four  steps: 
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1.  Planning:  During  the  planning  stage,  the  needs  of  eombatant  eommands 
are  analyzed  and  the  findings  are  published  in  the  Joint  Programming 
Guidanee  doeument,  whieh  guides  the  DOD  eomponents’  efforts  to 
propose  aequisition  programs. 

2.  Programming:  During  the  programming  stage,  proposed  programs  are 
fleshed  out,  and  the  Program  Objeetive  Memorandum,  a  doeument  that 
outlines  the  antieipated  missions  and  objeetives  of  the  proposed  weapon 
system  and  antieipated  budget  requirements,  is  submitted  to  propose  these 
programs.  The  memoranda  are  reviewed  and,  as  deemed  appropriate, 
integrated  into  an  overall  defense  program. 

3  Budgeting:  Budgeting  oeeurs  eoneurrently  with  the  programming  stage. 
Proposed  budgets  are  reviewed  in  a  different  manner  than  proposed 
programs.  Upon  eompletion  of  a  program  deeision  or  as  a  result  of  a 
budget  review,  program  budget  deeisions  are  issued. 

4.  Execution:  Execution  occurs  simultaneously  with  the  program  and  budget 
reviews.  During  execution,  programs  are  evaluated  and  measured  against 
established  performance  metrics,  including  the  rates  of  funding  obligation 
and  expenditures.  (Schwartz,  2013) 

Much  of  acquisition  reform  results  from  discrepancies  in  what  occurred  during 
the  planning  and  programming  phases  when  compared  to  actual  budget  execution  within 
acquisition  programs. 

D,  THE  ACQUISITION  PROCESS  “LITTLE  A” 

The  U.S.  Department  of  Defense  (DOD)  is  by  far  the  largest  and  most  complex 

business  organization  in  the  world.  It  operates  more  than  5,400  installations 

worldwide  and  executes  more  than  15  million  contracts  per  year.  It  also  develops 

and  produces  the  most  sought-after  weapons  and  equipment  in  the  free  world. 

(Fox,  2011,  p.  1) 

The  DOD  is  complex,  and  the  acquisition  process  is  one  of  the  most  complex 
processes  existing  in  the  DOD.  Figure  2  portrays  the  acquisition  process,  its  phases  and 
milestones.  While  the  figure  seems  simple,  the  workings  of  the  process  are  complex. 
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•  The  Materiel  Development  Decision  precedes 
entry  into  any  phase  of  the  acquisition 
management  system 

•  Entrance  criteria  met  before  entering  phase 

•  Evolutionary  Acquisition  or  Single  Step  to 
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Figure  2.  Acquisition  Process  (from  DAU,  n.d.a) 


The  Material  Development  Decision  initiates  the  Material  Solution  Analysis 
(MSA)  phase  of  the  acquisition  process  after  a  user  need  is  identified.  The  goal  of  this 
phase  is  to  determine  the  method  (make  or  buy)  required  to  meet  end-user  needs.  This 
phase  includes  the  Analysis  of  Alternatives,  where  acquisition  officials  consider  any 
capability,  including  commercial  off-the-shelf,  which  can  satisfy  the  requirement  with  the 
greatest  benefit  regarding  cost,  schedule,  and  performance.  Emphasis  during  this  phase  is 
on  adequately  identifying  user  needs  so  that  there  are  no  requirement  misunderstandings 
in  the  early  stages  of  program  consideration.  The  program  moves  to  Milestone  A  with 
the  creation  of  the  Technology  Development  Strategy,  as  it  postures  for  progression  into 
the  Technology  Development  (TD)  phase.  The  Milestone  Decision  Authority  (MDA) 
evaluates  the  program  for  advancement  into  the  TD  phase  (DOD,  2013b). 

The  goal  of  the  TD  phase  is  to  produce  technologies  mature  and  capable  enough 
to  support  program  requirements.  The  acquisition  strategy — the  framework  for  the  entire 
program  acquisition  process — is  developed  during  this  phase  and  includes  developing  the 
system’s  cost,  schedule,  and  performance  criteria.  Product  prototypes  are  developed  to 
demonstrate  initial  capabilities  required  to  move  into  the  following  acquisition  process  phase. 
Requests  for  proposals  are  disseminated,  and  contractors  begin  to  compete  for  the  contract 
award.  The  program  manager  (PM)  develops  the  preliminary  design  review,  which 
establishes  baselines  for  human,  software,  and  hardware  support  systems.  At  Milestone  B, 
the  MDA  determines  whether  to  allow  a  program  to  advance  into  the  Engineering  and 
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Manufacturing  Development  (EMD)  phase  based  on  technologieal  eapabilities  demonstrated 
and  teehnological  risk  redueed  during  the  ID  phase  (DOD,  2013b). 

A  program  advanees  into  the  EMD  phase  if  the  teehnology  has  matured,  the  MDA 
has  approved  the  aequisition  strategy,  and  funding  is  fully  approved.  The  intent  of  the 
EMD  phase  is  to  demonstrate  sound  manufaeturing,  logistieal,  and  end-product 
capabilities.  Collaboration  with  end  users  during  the  prior  two  aequisition  phases  proves 
imperative  beeause  in  those  phases,  key  performanee  parameters  (KPPs)  are  identified  as 
the  baseline  by  which  the  produet  will  be  analyzed.  Signifieant  testing  oecurs  during  this 
phase,  and  produets  must  demonstrate  their  eapabilities  in  a  relevant  developmental 
environment.  The  EMD  phase  ineorporates  all  manufaeturing,  industrial,  logistieal,  and 
human  systems  into  the  program,  and  these  systems  must  demonstrate  their  eapability 
prior  to  the  next  milestone  deeision.  When  the  program  has  demonstrated  these 
eapabilities  and  sueeessfully  eompleted  developmental  test  and  evaluation  (DT&E),  the 
MDA  eonduets  the  Milestone  C  and  low-rate  initial  production  (ERIP)  analysis,  which 
determines  whether  the  program  advanees  to  the  Production  and  Deployment  phase 
(DOD,  2013b). 

The  DOD  5000.02  states  that  “the  purpose  of  the  Produetion  and  Deployment 
phase  is  to  aehieve  an  operational  eapability  that  satisfies  mission  needs”  (DOD,  2013b, 
p.  27).  Operational  test  and  evaluation  tests  the  produet  or  system  in  an  operational 
environment  (OE),  ineorporating  end  users  and  support  systems  equivalent  to  the  OE. 
After  the  system  demonstrates  capacity  to  meet  operational  needs  and  meets  other 
Milestone  C  eriteria  mentioned  in  DOD  Instruetion  (DODI)  5000.02  Enelosure  9,  the 
ERIP  deeision  allows  initial  produetion  to  begin.  Key  to  this  phase  is  the  reetifieation  of 
any  diserepaneies  identified  during  the  previous  phase,  whieh  ensures  that  issues 
identified  during  T&E  do  not  persist  in  the  ERIP.  Eurthermore,  as  manufaeturing, 
logistieal,  and  industrial  systems  prove  their  capability  to  produce  the  initial  low  rates  of 
produetion,  these  systems  are  further  assessed  to  determine  their  eapacity  to  move  into 
full-rate  produetion  (ERP).  ERP  leads  into  the  full  deployment  of  the  product  or  system 
to  the  intended  end  user  (DOD,  2013b). 
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The  final  phase  of  the  aequisition  proeess  is  the  Operations  and  Support  (OS) 
phase.  Prior  to  entering  the  OS  phase,  a  program  must  demonstrate  that  it  has  a  life-eycle 
sustainment  plan  (LCSP),  among  other  items.  The  LCSP  is  part  of  the  program  from  the 
beginning  and  develops  in  eonjunction  with  the  maturing  program,  ensuring  that  support 
responsibilities  are  adequately  defined  and  that  trained  personnel,  equipment,  and 
infrastrueture  fulfilling  support  and  maintenanee  roles  are  present.  This  phase  spans  the 
remaining  life  cyele  of  the  program,  from  FRP  to  the  end  of  the  produet  or  system’s  life. 
The  program  offiee  eontinues  to  monitor  the  produet  or  system’s  performanee  data, 
determining  whether  adjustments  must  be  made  to  units  in  production  (DOD,  2013b). 

E,  THE  JOINT  READINESS  OVERSIGHT  COUNCIL  AND  JCIDS 

What  drives  a  requirement?  Jones  and  McCaffery  (2008)  may  have  stated  it  best 
when  they  said,  “Threats  to  national  security,  perceived  or  actual,  and  political  priorities 
drive  the  amount  of  defense  funding  requested  and  appropriated  for  weapons  acquisition” 
(p.  83).  The  Chairman  of  the  Joint  Chiefs  of  Staff  leads  the  Joint  Readiness  Oversight 
Council  (JROC)  through  the  JCIDS  process  and  the  JROC’s  integration  into  the 
acquisition  process  It  is  imperative  that  the  JROC  be  involved  with  the  acquisition 
process  from  the  moment  that  an  end-user  requirement  is  identified.  This  involvement 
ensures  that  programs  do  not  suffer  from  redundancies  if  a  capability  exists  in  a  separate 
military  branch,  allowing  for  more  efficient  allocation  of  procurement  funds.  “JCIDS 
provides  a  transparent  process  that  allows  the  JROC  to  balance  joint  equities  and  make 
informed  decisions  on  validation  and  prioritization  of  capability  requirements”  (DAU, 
n.d.b,  p.  1).  The  JROC  and  JCIDS  fulfill  many  key  roles  within  the  acquisition  process, 
as  highlighted  in  the  purple  portions  of  Figure  3. 
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Continuous Tedinoto^y  Development  and  Maturation 


Figure  3.  JCIDS  Review  Proeess  (from  DOD,  2013b) 

DOD  eomponents  and  serviees  eonduct  eapabilities-based  assessments  that  lead 
to  a  determination  of  whether  a  eapability  gap  exists.  JCIDS  plays  a  key  role  by 
inereasing  eollaboration  among  serviees.  When  a  eapability  gap  is  determined,  an  initial 
eapabilities  doeument  (ICD)  is  generated  if  non-material  solutions  have  been  exhausted. 
Non-material  solutions  are  analyzed  on  the  basis  of  doetrine,  organization,  training, 
materiel,  leadership,  personnel,  faeilities,  and  poliey  (DOTMLPF-P).  The  JROC  assists 
with  determining  whether  the  requirement  ean  be  met  by  ehanging  an  aspeet  of 
DOTMLPF-P;  otherwise,  an  ICD  is  developed.  The  ICD  is  the  first  key  doeument  that 
JCIDS  eontributes  to  the  aequisition  system.  This  doeument  feeds  into  the  MSA  and  the 
Coneept  Exploration  phase  (Chairman  of  the  Joint  Chiefs  of  Staff  [CJCS],  2012). 

JCIDS  and  JROC  involvement  in  the  aequisition  proeess  remains  imperative  to 
meeting  user  needs.  JCIDS  develops  eapability  development  doeuments,  whieh  verify 
that  the  developing  product,  system,  and  technology  are  capable  of  moving  into  the  next 
phase  of  the  acquisition  process.  The  JROC  regularly  interfaces  with  acquisition’s 
representatives  and  combatant  units,  verifying  KPPs  and  ensuring  that  the  program  is  on 
track  with  the  needs  of  joint  forces  and  cost,  schedule,  and  performance  parameters. 
JCIDS  is  also  responsible  for  developing  the  capabilities  production  documents,  which 
indicate  that  the  program’s  producability  and  logistical  and  infrastructural  support 
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systems  are  mature  and  eapable  enough  for  the  program  to  exit  one  phase  and  enter  the 
next  (DOD,  2013b). 

This  eomplieated  proeess,  eomposed  of  multiple  organizations  and  many 
stakeholders,  has  seen  many  reformation  initiatives  and  regulations  over  the  years. 
Intended  benefits  of  reform  varied  from  a  more  streamlined  proeess,  higher  end-user 
satisfaction,  and  control  of  cost  and  schedule  overruns.  Prior  to  understanding  why 
reform  has  occurred  and  whether  reform  actions  have  been  successful,  one  must 
understand  the  process,  its  complexities,  and  the  differing  incentives  driving  stakeholders 
in  and  around  the  acquisition  process. 

F.  PERCEPTIONS  REGARDING  ACQUISITION  IMPLEMENTATION 

Acquisition  reform  dates  back  to  as  early  as  1862  when  President  Abraham 
Lincoln  requested  the  resignation  of  the  secretary  of  war,  Simon  Cameron,  due  to 
contracting  “corruption  and  mismanagment”  (Schwartz,  2013,  p.  12).  Other  acquisition 
reforms  continued  in  the  decades  that  followed,  including  in  1958,  when  the  Defense 
Reorganization  Act  further  restructured  the  acquisition  process.  Modem  acquisition 
reform  initiatives  date  back  to  the  early  1990s  and  encompass  a  list  of  63  initiatives 
(Hanks,  Axelband,  Lindsay,  Malik,  &  Steele,  2005).  More  recent  efforts  led  to  overhauls 
of  major  defense  acquisition  policy,  including  the  acquisition  operations  manual  DODl 
5000  series.  However,  despite  many  efforts  to  reform  the  acquisition  system,  multiple 
programs  continued  to  fail  at  meeting  cost  and  schedule  requirements  (see  Figure  4). 
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Fiscal  Year 

2003 

2007 

2008 

Portfolio  size 

Number  of  programs 

77 

95 

96 

Total  planned  commitments 

$1.2  trillion 

$1.6  trillion 

$1.6  trillion 

Commitments  outstanding 

$724.2  billion 

$875.2  billion 

$786.3  billion 

Portfolio  Indicators 

Change  to  total  RDT&E  costs  from  first  estimate 

37  percent 

40  percent 

42  percent 

Change  to  total  acquisition  cost  from  first 
estimate 

19  percent 

26  percent 

25  percent 

Total  acquisition  cost  growth 

$183  billion 

$301.3  billion* 

$296.4  billion 

Share  of  programs  with  25  percent  increase  in 
program  acquisition  unit  cost  growth 

41  percent 

44  percent 

42  percent 

Average  schedule  delay  in  delivering  initial 
capabilities 

18  months 

21  months 

22  months 

Source:  GAO  analysis  of  DoD  data. 


Notes;  Data  was  obtained  from  DoD’s  Selected  Acquisition  Reports  (SAR)  (dated  December 
2002,  2006,  and  2007).  In  a  few  cases,  data  were  obtained  directly  from  program  offices.  The 
number  of  programs  reflects  the  programs  with  SARs;  however,  in  our  analysis  we  have  broken 
a  few  SAR  programs  into  smaller  elements  or  programs.  Not  all  programs  had  comparable  cost 
and  schedule  data  and  these  programs  were  excluded  from  the  analysis  where  appropriate. 
Portfolio  performance  data  do  not  include  costs  of  developing  Missile  Defense  Agency 
elements  or  the  Defense  Integrated  Military  Human  Resources  System  (DIMHRS)  program. 
The  total  acquisition  cost  growth  for  the  2007  portfolio  was  $295  billion  in  2008  constant  dollars 
(GAO-09-663T.  p.  0). 

Figure  4.  Analysis  of  DOD  MDAP  Portfolios  (from  Francis,  2009) 

These  failures  to  meet  requirements  led  to  another  round  of  acquisition  reforms, 
including  the  WSARA  of  2009  and  the  Better  Buying  Power  initiatives.  These  reforms 
again  intended  to  reduce  cost  and  schedule  overruns  while  improving  performance. 

G.  CONCLUSION 

The  number  of  initiatives  spanning  approximately  23  years  of  acquisitions — 63 
initiatives  in  total — is  one  of  the  key  points  from  the  RAND  study  conducted  by  Hanks  et 
al.  (2005).  These  initiatives  are  often  mere  policy  documents  generated  as  a  new  leader 
assumes  position  within  the  purview  of  the  acquisition  process  and  rarely  find  their  way 
into  actual  acquisition  regulation.  Another  issue  plaguing  acquisitions  and  reform  efforts 
is  discussed  in  a  more  recent  study  conducted  by  the  Government  Accountability  Office 
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(Francis,  2013).  This  study  concluded  that  the  ineentives  driving  multiple  stakeholders 
involved  with  the  aequisition  proeess  differ  vastly  and  eontinue  to  frustrate  the 
aequisition  system.  Our  researeh  analyzes  ineentives  and  how  eurrent  reward  and 
evaluation  systems  may  not  be  aligned  with  regulation  implementations  (sueh  as  the 
WSARA  of  2009)  and  ereate  a  diseonneet  between  regulation  and  aetivities  in  an 
aequisition  organization. 
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II.  LITERATURE  REVIEW  AND  WSARA  BACKGROUND 


In  Chapter  I,  we  established  the  foundation  of  this  research.  In  this  chapter,  we 
provide  a  literature  review  surrounding  the  WSARA.  This  review  covers  the  following: 

•  establishment  and  implementation  of  the  WSARA, 

•  WSARA  implementation  and  program  and  product  successes, 

•  desired  effects  are  occurring  from  the  WSARA,  and 

•  cost,  schedule,  and  performance  and  contract  metrics. 

To  capture  the  complex  timeline  of  reform  initiatives,  the  paragraphs  in  this 
chapter  are  organized  by  the  project’s  objectives;  resources  are  described  in 
chronological  order. 

A.  LITERATURE  SELECTION 

The  literature  surrounding  acquisition  reform  helps  to  illuminate  and  determine 
the  project  objectives.  To  determine  whether  there  is  a  correlation  between  WSARA 
implementation  and  program  and  product  success,  we  first  examine  the  need  for 
establishment  of  the  WSARA  and  the  key  aspects  of  acquisition  reform.  Next,  to 
determine  whether  WSARA  legislation  is  having  the  desired  effects,  we  address  current 
legislative  debate  and  articles.  Finally,  to  assess  the  effects  of  WSARA  legislation  on 
cost,  schedule,  performance,  and  contract  metrics,  we  look  at  the  WSARA’s  intended 
effects,  perceived  effects,  and  actual  effects.  We  also  show  how  effects-based 
management  theory  can  be  tied  to  WSARA  implementation  and  describe  why  there  is  a 
need  for  an  accurate  assessment  on  the  measures  of  performance  related  to  the  WSARA. 

We  selected  the  literature  because  there  is  not  much  data  on  the  WSARA  or  its 
actual  effects.  We  chose  a  variety  of  mediums  due  to  the  ambiguity  surrounding  the 
topic.  To  see  more  deeply  into  the  subject  matter,  we  selected  journal  articles  and  books 
from  experts  on  acquisition  reform.  We  also  selected  key  GAO  reports  to  illustrate 
ongoing  challenges  and  analysis.  We  also  chose  congressional  transcripts  that  cover 
acquisition  reform  and  some  debate  on  the  WSARA.  And  finally,  we  selected  relevant 
news  articles  to  reflect  public  understanding  and  interpretation  of  the  legislation  and 
acquisition  reform  issues. 
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1,  Establishing  a  Need  for  the  WSARA 

The  WSARA  was  a  necessary  piece  of  legislation.  Its  intent  was  “to  improve  the 
organization  and  procedures  of  the  Department  of  Defense  for  the  acquisition  of  major 
weapon  systems,  and  for  other  purposes”  (Weapon  Systems  Acquisition  Reform  Act 
[WSARA]  of  2009).  Decades  of  acquisition  reform  and  initiatives  led  up  to  the 
implementation  of  the  law.  Hanks  et  al.  (2005)  identified  63  acquisition  reform 
initiatives  from  the  1960s  to  2001  that  were  aimed  at  streamlining  logistics;  improving 
the  industrial  base;  managing  total  life-cycle  system  management;  and  reducing  waste, 
fraud,  and  abuse. 

The  Military  Reform  Caucus  (MRC),  cofounded  in  part  by  Newt  Gingrich,  was 
originally  founded  in  1981  (Malishenko,  1987).  Malishenko  (1987)  highlighted 
congressional  dynamics  in  his  article  with  the  MRC.  It  sheds  light  on  early  attempts  for 
bipartisanship  between  political  parties  of  the  House  and  Senate  in  the  1980s  during  the 
early  years  of  the  acquisition  reform  initiatives.  The  MRC  membership  was  based  on 
interested  congressional  members  and  helped  lay  the  foundation  for  future  legislation 
(Malishenko,  1987). 

Thompson  (1992)  described  his  frustration  with  the  number  of  laws  and  extra 

layers  of  bureaucracy  added  to  the  DOD  acquisition  processes  and  stated  that  “greatly 

increased  regulation  and  oversight  [are]  needed  to  fix  the  system”  (p.  729).  Furthermore, 

Congress  “added  an  assortment  of  new  laws  aimed  at  further  disciplining  DOD 

acquisitions  to  the  existing  1,150  feet  of  legislation  and  case  law  governing  procurement” 

(Thompson,  1992,  p.  729).  Thompson  (1992)  highlights  that 

most  acquisition  experts  call  for  the  elimination  of  the  statutes  that  limit  the 
proper  exercise  of  the  acquisition  officer’s  discretion.  They  all  call  for  a 
substantial  reduction  in  the  size  and  scope  of  the  systems  commands,  systems 
specifications,  evaluation  teams,  and  in  the  number  of  criteria  the  evaluation 
teams  consider  in  the  source  selection  process.  Most  call  for  clear  command 
channels,  limited  reporting  requirements,  and  project  teams  with  small  high- 
quality  staffs.  (Thompson,  1992,  p.  744) 

Many  reform  initiatives  were  established  leading  up  to  the  WSARA  law.  Most 
notable  is  President  Reagan’s  Packard  Commission  report  from  1986.  The  Packard 
Commission  “concluded  that  the  primary  problems  with  the  acquisition  process  were  the 
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same  ones  identified  in  previous  deeades  (eost  growth,  schedule  delays,  performance 
shortfalls)”  (Christensen,  Stearic,  &  Vickery,  1999,  p.  251).  Through  their  study, 
Christensen  et  al.  (1999)  highlighted  previous  attempts  of  legislation  that  were  not 
adequate  in  improving  cost  performances:  “Despite  the  implementation  of  more  than  two 
dozen  regulatory  and  administration  initiatives,  there  has  been  no  substantial 
improvement  in  the  cost  performance  of  defense  programs  for  more  than  30  years” 
(p.  252). 

A  2005  RAND  report  (Hanks  et  ah,  2005)  highlighted  acquisition  reform 
initiatives  that  illustrate  63  initiatives  from  1966  up  to  2001  that  were  marginally 
successful.  Several  of  the  interview  quotations  from  the  RAND  report  emphasized  this 
theme: 

•  “Acquisition  Reform  inhibitors:  need  to  change  law  or  just  working  the 

edges;  need  to  change  how  funding  is  done.  Example:  _  program 

subsidizes  everything  at  _  (Major  Subordinate  Command),  so  the 

incentives  are  to  keep  selling  parts  [to  that  program]  and  not  let  them  fix 
the  parts”  (program  executive  officer-military;  p.  121); 

•  “The  test  community  is  still  focused  on  their  reporting  requirements  rather 
than  testing  to  fix”  (PM-military;  p.  110). 

In  their  analysis  of  organizational  transformation,  Eide  and  Allen  (2012) 

explained  acquisition  reform  legislation  in  clear  and  concise  detail.  Starting  with  the 

Packard  Commission  report  of  1986,  Eide  and  Allen  (2012)  specifically  stated. 

Excellence  in  defense  management  will  not  and  cannot  emerge  by  legislation  or 
directive.  Excellence  requires  the  opposite — responsibility  and  authority  placed 
firmly  in  the  hands  of  those  at  the  working  level,  who  have  knowledge  and 
enthusiasm  for  the  tasks  at  hand  (p.  102). 

The  Goldwater-Nichols  Act  of  1986  followed  the  Packard  Commission  report 
and  addressed  some  of  the  changes  that  the  Packard  Commission  suggested,  specifically 
“diluted  authority  for  execution”  (Eide  &  Allen,  2012,  p.  102).  Moving  up  a  decade  into 
the  1990s,  a  more  business-minded  aspect  became  apparent  in  legislation  and  reform 
initiatives.  Eocusing  on  the  acquisition  workforce,  “The  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA)  of  1990  addressed  the  need  to  improve  the  quality  of  the 
acquisition  workforce,  establishing  formal  career  paths  and  standards  for  education  and 
training”  (Eide  &  Allen,  2012,  p.  102). 
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President  Clinton  signed  two  laws  focused  on  acquisition  reform  during  his 
presidency:  the  Federal  Acquisition  Streamlining  Act  (FAS A)  of  1994  and  the  Clinger- 
Cohen  Act  of  1996.  To  broaden  the  definition  of  what  a  commercial  item  was,  “FASA 
streamlined  procurement  of  commercial  items  by  exempting  them  from  existing  laws” 
(Fide  &  Allen,  2012,  p.  102).  The  Clinger-Cohen  Act  of  1996  facilitated  the  way  that 
the  government  acquires  information  technology  by  “eliminat[ing]  cost  accounting 
standards  that  had  discouraged  commercial  companies  from  doing  business  with  the 
federal  government”  (Fide  &  Allen,  2012,  p.  102). 

In  2005,  the  Defense  Acquisition  Performance  Assessment  Project  was 
established  to  provide  an  assessment  of  the  DOD’s  acquisition  system.  Major  findings  of 
the  project  included  identification  of  “excessive  oversight  and  complex  acquisition 
processes  as  cost  and  schedule  drivers,  and  called  for  stability  of  requirements  as  an 
essential  element  for  an  effective  acquisition  system”  (Fide  &  Allen,  2012,  p.  104). 

The  WSARA  aimed  at  improving  the  success  of  major  defense  acquisition 
programs  (MDAPs).  To  accomplish  this  task,  the  WSARA  provided  focus  on  early 
decision  variables  such  as  “reliable  and  independent  baseline  cost  estimates,  rigorous 
early  developmental  testing  and  systems  engineering  oversight,  and  strong  gatekeeping  to 
prevent  programs  from  proceeding  with  too  much  risk  of  immature  technology”  (Fide  & 
Allen,  2012,  p.  106).  Along  with  program  success,  an  implied  goal  of  the  WSARA  was 
to  reduce  costs  and  risk.  Increased  competition  was  incorporated  in  the  WSARA  to  help 
accomplish  this  goal  (Fide  &  Allen,  2012,  p.  106). 

The  WSARA’s  key  provisions  altered  the  DOD’s  acquisition  bureaucracy  in 
addition  to  process  systems  and  senior  management  oversight  changes  (Berteau, 
Hofbauer,  &  Sanok,  2010,  p.  4).  The  WSARA  also  legislated  important  changes  on 
acquisition  processes.  According  to  Berteau  et  ah,  WSARA 

•  increased  competition  throughout  the  acquisition  process, 

•  improved  requirement  formulation  processes, 

•  improved  cost-estimation  processes, 

•  a  more  stringent  set  of  regulations  on  organizational  conflicts  of  interest. 


revised  Milestone  A  and  Milestone  B  certification  processes, 
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•  revised  processes  for  reporting  critical  cost  growth,  and 

•  increased  congressional  oversight  through  heightened  reporting 
requirements 

The  WSARA  created  four  senior-level  Office  of  the  Secretary  of  Defense 
positions  with  military-service-level  equivalents  for  the  DOD  (Berteau  et  ah,  2010; 
WSARA  of  2009): 

•  Director  of  Cost  Assessment  &  Program  Evaluation  (DCAPE) 

•  Director,  Development  Test  &  Evaluation  (DT&E) 

•  Director,  Systems  Engineering  (SE) 

•  Director  for  Performance  Assessments  and  Root  Cause  Analyses 
(PARCA) 

The  primary  responsibilities  of  these  offices  are  illustrated  in  Eigure  5. 


Table  2:  Key  Responsibilities  of  Offices  Established  as  a  Result  of  the  Reform  Act 


Office 

Primary  responsibilities 

Systems  Engineering 

serves  as  principal  advisor  to  the  Secretary  of  Defense  and  the  Under  Secretary  of  [>efense 

for  Acquisition,  Technology  and  Logistics  on  systems  engineering  activities  in  the  department 

develops  systems  engineering  and  development  planning  guidance  for  DOD 

reviews  and  approves  major  defense  acquisition  program  systems  engineering  plans 

monitors  major  defense  acquisition  program  systems  engirteering  activities 

reports  to  Congress  annually  on  systems  engineering  organization,  capabilities,  and  activities 

Developmental  Test  and 
Evaluation 

serves  as  principal  advisor  to  the  Secretary  of  Defense  and  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology  and  Logistics  on  developmental  test  and  evaluation  activities 
develops  developmental  test  and  evaluation  guidance  for  DOD 

reviews  and  approves  major  defense  acquisition  program  developmental  test  and  evaluation 
plans 

monitors  and  reviews  acquisition  program  developmental  test  and  evaluation  activities  of 
major  defense  acquisition  programs 

reports  to  Congress  annually  on  developmental  test  and  evaluation  organization,  capabilities, 
and  activities 

Cost  Assessment  and  Program 
Evaluation 

serves  as  principal  advisor  to  the  Secretary  of  Defense  and  other  senior  officials  on  matters 
related  to  cost  analysis  and  the  planning  and  programming  phases  of  the  planning, 
programming,  budgeting,  and  execution  system 

develops  independent  cost  estimates  for  major  defense  acquisition  programs  prior  to  major 
milestone  decisions  and  updates  Independent  cost  estimates  when  programs  have  exceeded 
critical  cost  thresholds,  known  as  Nunn  McCurdy  breaches 

reviews  existing  systems  and  methods  for  tracking  and  assessing  operation  and  support 
costs  on  major  defense  acquisition  programs 

develops  analysis  of  alternative  study  guidance  for  major  defense  acquisition  programs 
approves  the  analysis  of  alternatives  study  plan  for  each  major  defense  acquisition  program 

Performance  Assessments  and 
Root  Cause  Analyses 

assesses  major  acquisition  program  performance  through  independent  analyses  and  through 
the  Defense  Acquisition  Executive  Summary  process 

identifies  the  root  causes  of  cost  growth  and  other  problems  on  programs  that  experience  a 
critical  Nunn  McCurdy  cost  breach 

Souicb:  GAO  AnalyM  of  Weapon  Syslens  Acqweon  Reform  Ad  of  2009. 


Figure  5.  WSARA  Key  Office  Responsibilities  (from  Sullivan,  2012,  p.  5) 
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2. 


WSARA  Implementation  and  Correlation  to  Program  and  Product 
Success 


Finding  the  relationship  between  a  legislation’s  intended  effeets  and  its  aetual 
results  on  the  group  to  be  legislated  is  ehallenging.  Legislation  rarely  has  a  direet  visual 
result  on  the  DOD’s  bottom  line  but  rather  ereates  a  waterfall  of  ehanges  and  seeondary 
effeets.  A  eorrelation  between  two  variables  ean  exist,  but  eausation  is  a  different  matter. 
In  our  research  of  the  WSARA  law  and  its  provisions,  we  make  an  assumption  that  the 
WSARA  is  the  initiator  of  bottom-line  change,  if  not  the  catalyst  for  changes  that  occur.  . 

The  bottom  line  in  the  corporate  world  is  profit.  For  a  governmental  entity  like 
the  DOD,  the  acquisition  bottom  line  is  measured  in  cost,  schedule,  and  performance. 
There  is  a  correlation  between  other  acquisition  initiatives  and  cost,  schedule,  and 
performance  metrics.  Biery  (1992),  along  with  some  other  researchers,  argued  that  the 
DOD  was  doing  well  during  the  1990s,  positing  that  “if  meeting  cost  and  schedule 
objectives  measures  an  institution’s  managerial  efficiency,  then  the  DOD  is  more 
efficient  than  the  typical  commercial  business  or  government  organization”  (p.  644). 
Biery  (1992)  also  illustrated  that  the  end  of  each  decade  has  a  decreasing  trend  of  cost 
growth  and  compound  annual  growth  rate. 

Countering  Biery’ s  (1992)  argument,  Christensen  et  al.  (1999)  conducted  a  cost 
overrun  analysis  on  269  contracts  that  were  four  years  before  and  four  years  after  the 
implementation  of  the  Packard  Commission  recommendations.  The  assumption  was  that 
there  would  be  a  clear  drop  in  cost  overruns.  However,  Christensen  et  al.’s  (1999) 
results,  as  seen  in  Figure  6,  “show  that  the  Packard  Commission’s  recommendations  did 
not  improve  the  cost  performance  of  defense  acquisition  contracts”  (p.  256). 
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Cost  Performance 


Contracts 


■  Before 

■  After 


Figure  6.  Graph  from  Cost  Overrun  Analysis  on  Paekard  Commission  Reforms 

(from  Christensen  et  ah,  1999) 


The  WSARA’s  implementation  has  had  suceess,  as  illustrated  in  the  third  GAO 
report  on  the  WSARA,  authored  by  Sullivan  (2012).  Aeeording  to  the  report,  the 
“GAO’s  analysis  of  11  weapon  acquisition  programs  showed  the  Reform  Act  has 
reinforced  early  attention  to  requirements,  cost  and  schedule  estimates,  testing,  and 
reliability”  (Sullivan,  2012,  p.  1).  This  is  reinforced  by  GAO  Report  No.  GAO-14-145T, 
authored  by  Francis  (2013);  Francis  indicated  that  “to  the  extent  reforms  like  the  Weapon 
Systems  Acquisition  Reform  Act  and  DOD’s  Better  Buying  Power  initiatives  are  being 
implemented,  they  are  having  a  positive  effect  on  individual  programs”  (p.  5). 

3,  Desired  Effects  From  the  WSARA 

The  Department  of  Defense’s  Rapid  Acquisition  Process:  Is  It  a  Model  for 
Improving  Acquisition,  the  first  hearing  before  the  Panel  on  Defense  Acquisition  Reform 
of  the  House  Armed  Services  Committee  on  June  3,  2009,  highlighted  ongoing 
challenges  associated  with  the  DAS.  During  the  hearing,  Gordon  England,  a  former 
deputy  secretary  of  defense  and  president  of  the  company  E6  Partners,  spoke  on  the 
complexity  of  the  DAS: 
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Counterintuitively,  that  means  you  want  to  give  managers  more  flexibility. 

The  more  eomplex  the  system,  the  more  flexibility  you  need,  managers 
need.  The  trend  is  always  the  other  way.  That  is  it  gets  more  complex,  we 
add  layers  of  bureaucracy  and  regulation  and  control  and  that  makes  it 
almost  impossible  to  run  very  complex  programs.  So  the  system  today  is 
way  overburdened.  It  is  over-burdened  by  the  Department,  it  is 
overburdened  by  the  Congress.  As  it  becomes  a  more  complex  system,  we 
need  to  simplify  it,  otherwise  managers  won't  be  able  to  operate.  {The 
Department  of  Defense ’s  Rapid  Acquisition  Process,  2009,  p.  6) 

Representative  Jim  Cooper  from  Tennessee  mentioned  additional  complexity 
added  to  the  acquisition  process  from  the  WSARA  but  unfortunately  suggested  that  no 
one  read  the  law,  stating,  “I  am  worried  about  the  Tower  of  Babel  effect  when  we  create 
a  system  that  is  so  complex  that  nobody  can  understand  it.  We  were  just  joking  prior  to 
the  hearing  that  how  many  people  actually  read  the  weapons  acquisition  bill  that  we  just 
passed.  Nobody”  {The  Department  of  Defense’s  Rapid  Acquisition  Process,  2009,  p.  17). 
Robert  Andrews,  the  chairman  of  the  panel  at  the  time,  addressed  the  lack  of  coordination 
between  requirements,  budgeting,  and  procurement  {The  Department  of  Defense’s  Rapid 
Acquisition  Process,  2009,  p.  10).  Luckily,  the  WSARA  addressed  some  of  Andrews’ 
concerns. 

Programs  are  just  beginning  to  implement  the  provisions  of  the  WSARA 
(Sullivan,  2012).  The  report  points  to  little  to  no  improvement  in  acquisition  areas. 
However,  according  to  a  2013  report  by  the  Government  Business  Council,  there  has 
been  improvement  in  personal  accountability  (as  shown  in  Figure  7). 
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Figure  7.  Acquisition  Areas  of  Improvement,  Affirmative  Answers 
(from  Government  Business  Council,  2013,  p.  21) 


4.  Conclusion 

This  chapter  introduced  past  and  current  literature  on  the  establishment  and 
implementation  of  the  WSARA;  correlation  between  WSARA  implementation  and 
program  and  product  successes;  intended  effects  of  the  WSARA;  cost,  schedule,  and 
performance  and  contract  metrics;  and  effects-based  management.  The  next  chapter 
describes  research  methodology,  our  data  collection  process,  a  description  of  the  Program 
Executive  Office  for  Combat  Support  and  Combat  Service  Support  (PEO  CS&CSS),  a 
list  of  our  survey  questions,  and  the  analysis  we  conducted. 

The  literature  covering  this  topic  has  several  agreements.  Eirst,  consistent 
acquisition  reform  is  needed  as  requirements  are  continuously  generated  from  threat- 
based  analysis  within  budgetary  constraints;  refinement  and  reform  of  the  acquisition 
system  is  needed.  Second,  acquisition  reform  is  ongoing.  As  mentioned,  threats  and  the 
operating  environment  are  always  changing.  Therefore,  timely  and  accurate  reform  is 
needed  in  legislation  and  policies  to  develop  the  right  solution  at  the  right  time.  Third, 
flexibility  is  needed  in  a  complex  system.  The  ability  to  provide  resources  to  the  right 
people  to  execute  solutions  in  a  complex  system  is  paramount.  Eifth,  more  accurate 
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metrics  are  needed  on  cost,  performance,  and  schedule  to  provide  senior  leadership  with 
the  right  information  to  make  decisions. 

The  literature  covering  this  topic  has  a  couple  of  disagreements.  First,  several  of 
the  authors  at  different  points  in  time  (both  current  and  past)  have  disagreed  on  whether 
acquisition  reform,  the  WSARA  included,  is  doing  a  good  job.  Hindsight  is  20/20 — it  is 
easier  to  measure  reforms  10  years  after  they  have  been  implemented  and  the  entire 
operating  environment  has  changed.  Second,  there  is  disagreement  on  how  success  is 
measured.  Managers  at  the  program  and  product  levels  have  a  different  interpretation  of 
which  metrics  should  be  used  versus  those  at  Congress  or  in  the  private  sector. 

Our  project  covers  a  specific  case  study.  We  focus  on  the  PEO  CS&CSS  and  its 
products.  By  focusing  on  a  single  office  and  seeing  how  the  WSARA  is  being 
implemented  there,  we  can  provide  a  more  accurate  sample  rather  than  providing  a 
blanket  population  or  randomly  selected  programs.  We  can  also  identify  successes  and 
challenges  more  accurately.  The  lessons  learned  at  a  specific  program  office  could  then 
be  utilized  at  other  offices  as  the  personnel  see  fit.  Furthermore,  we  use  an  analysis  on 
effects-based  management  and  how  it  could  be  implemented  into  different  levels  of 
management  to  yield  measurable  results. 
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III.  METHODOLOGY 


A,  INTRODUCTION 

In  this  chapter,  we  address  the  organizational  structure  of  the  PEO  CS&CSS  and 
then  explain  our  interview  and  quantitative  data  methodologies.  We  generated  interview 
questions  and  research  efforts  utilizing  the  primary  and  seeondary  research  questions  for 
this  project.  The  project  questions  are  as  follows: 

•  Overarching  project  question:  How  are  WSARA  Title  II  polieies  being 
applied  in  MDAPs? 

•  Why  was  the  WSARA  needed,  or  what  were  the  precursors  that  led  to  the 
WSARA? 

•  What  was  the  intended  effect  of  the  WSARA? 

•  How  effective  has  the  implementation  of  the  WSARA  been  in  the  PEO 
CS&CSS? 

•  Are  there  gaps  between  the  intent  and  application  of  Title  II  of  the 
WSARA  within  a  major  PEO?  If  so,  why? 

Our  intent  with  these  questions  was  to  determine  whether  acquisition  reforms, 
sueh  as  those  included  in  Title  II  of  the  WSARA  of  2009,  have  their  desired  effeet  in 
major  aequisition  programs. 

B,  PEO  CS&CSS  LOCATION  AND  ORGANIZATIONAL  STRUCTURE 

The  PEO  CS&CSS  is  located  within  the  Tank  and  Automotive  Command 
(TACOM)  headquarters  perimeter  in  Warren,  MI,  and  is  one  of  four  PEOs  within  the 
TACOM  structure.  The  PEO  CS&CSS  follows  the  typieal  major  aequisition 
organization:  the  program  executive  officer  and  his  or  her  deputies  orehestrate  the 
operations  of  the  PMs  (civilian  or  military  members  in  Grade  06  [colonel]  or  the  civilian 
equivalent)  and  produet  managers  (PdMs;  civilian  or  military  members  in  Grade  05 
[lieutenant  colonel]  or  the  civilian  equivalent)  and  their  various  assistants  and  staff.  Eor 
the  PEO  CS&CSS,  the  PM  level  is  broken  down  into  the  following  eategories: 

•  Eorce  proteetion:  This  eategory  includes  bridging,  combat  engineering, 
force  sustainment,  petroleum  and  water,  sets,  kits,  outfits  and  tools,  test 
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measurement  and  diagnostie  equipment,  and  eontingeney  basing 
infrastrueture; 

•  The  Joint  Program  Offiee  for  the  Joint  Light  Taetieal  Vehiele  (JLTV); 
This  eategory  ineludes  the  JLTV,  the  joint  Army-Marine  aequisition  effort 
eurrently  in  the  EMD  phase  of  the  aequisition  life  cyele; 

•  Mine-resistant,  ambush-pro teeted  vehieles  (MRAP):  This  eategory 
ineludes  MRAP-ATV  systems  and  joint  logisties; 

•  Mobile  eleetrie  power:  This  eategory  ineludes  small,  medium,  large,  and 
battery-powered  eleetrieal  power  sourees;  and 

•  Transportation  systems:  This  eategory  is  eomposed  of  personnel  and 
logisties  transportation  systems,  ineluding  medium  taetieal  vehieles, 
armored  seeurity  vehieles,  heavy  taetieal  vehieles,  allied  taetieal 
vehieles,  and  Army  watereraft  systems  (PEO  CS&CSS,  n.d.). 

C.  INTERVIEWEE  SELECTION 

In  late  Oetober  2013,  a  list  of  Army-sponsored  researeh  projeets  was  distributed 
through  the  Naval  Postgraduate  SehooTs  Graduate  Sehool  of  Business  and  Publie  Poliey 
ehannels.  Among  the  projeets  on  the  list  was  a  researeh  projeet  sponsored  by  the  PEO 
CS&CSS  stating  a  request  to  researeh  aequisition  reform  and  determine  whether  reforms 
are  implemented  and  the  intended  effeets  are  being  aehieved.  In  this  projeet,  we  foeused 
the  question  eoneerning  aequisition  reform  to  inelude  the  WSARA  of  2009,  speoifieally 
Title  IT  Title  II  of  the  WSARA  was  determined  to  be  the  portion  of  the  legislation  that 
was  most  readily  translatable  into  interview  and  researeh  questions  supporting  the  greater 
request  issued  by  the  PEO  CS&CSS. 

We  began  eorrespondenee  with  the  PEO  CS&CSS-designated  point  of  eontaet  to 
determine  interviewee  availability.  We  requested  individuals  operating  at  the  PM  and 
PdM  levels  as  the  primary  interviewees,  with  the  understanding  that  these  personnel  are 
ultimately  responsible  for  managing  the  eost,  sehedule,  performanee,  and  risk  of  PEO 
CS&CSS  programs.  Cost,  sehedule,  performanee,  and  risk  are  the  foeal  points  of  the 
Title  II  WSARA  reforms. 
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D,  INTERVIEW  METHOD 

We  conducted  interviews  on-site  with  the  PEO  CS&CSS  in  Warren,  MI.  We 
utilized  this  methodology  in  order  to  achieve  accurate  and  spontaneous  responses  from 
interviewees.  Rather  than  utilizing  an  e-mailed  survey  or  questionnaire,  which  potentially 
would  allow  interviewees  to  manipulate  or  research  a  best  answer  to  a  question,  face-to- 
face  interviews  afforded  honest  and  candid  responses  in  situ.  Additionally,  conducting 
interviews  on-site  allowed  us  to  develop  perceptions  concerning  the  PEO  CS&CSS  and 
its  priorities  that  could  not  be  gleaned  otherwise. 

E,  INTERVIEW  QUESTION  DEVELOPMENT 

We  developed  the  interview  questions  based  on  Title  II  of  the  WSARA  as  well  as 
expectancy  management  concepts.  Eirst,  we  converted  each  subsection  of  the  Title  II 
WSARA  legislation  into  the  form  of  a  question.  The  answers  to  the  question  would  serve 
as  a  means  for  qualifying  the  effects  of  the  WSARA  at  the  PEO  level.  We  then 
categorized  the  interview  questions  based  on  the  legislation  in  order  to  nest  them  for 
analysis  of  overarching  project  questions.  Second,  we  developed  questions  that  focused 
on  the  interviewee’s  interpretation  of  the  effectiveness  of  the  WSARA,  the  incumbent 
challenges  as  well  as  successes. 

F,  DEFENSE  ACQUISITION  MANAGEMENT  INFORMATION 

RETRIEVAL  SYSTEM  DATA 

The  Defense  Acquisition  Management  Information  Retrieval  System  (DAMIR) 
compiles  multiple  sources  from  MDAP  information  systems  as  well  as  major  automated 
information  systems.  According  to  its  website,  DAMIR  is  “the  authoritative  source  for 
Selected  Acquisition  Reports  (SAR),  SAR  Baseline,  Acquisition  Program  Baselines 
(APB),  and  Assessments”  (DAMIR,  2014). 

We  gathered  data  from  DAMIR  with  the  intent  of  conducting  a  high-level 
quantitative  analysis  on  the  data.  We  were  looking  for  a  correlation  between  program 
bottom-line  numbers  and  when  the  WSARA  was  implemented.  We  chose  SARs  for  all 


23 


Army  programs  present  in  the  DAMIR  system  from  2005  to  2012.  Aoeording  to  the 
press  release  (DOD,  2013a): 

SARs  summarize  the  latest  estimates  of  eost,  sehedule  and  performanee 
status.  These  reports  are  prepared  annually  in  eonjunetion  with  the 
submission  of  the  President’s  Budget.  ...The  total  program  eost  estimates 
provided  in  the  SARs  include  research  and  development,  procurement, 
military  construction,  and  acquisition-related  operation  and  maintenance. 

Total  program  costs  reflect  actual  costs  to  date  as  well  as  future 
anticipated  costs.  All  estimates  are  shown  in  fully  inflated  then-year 
dollars. 

The  SAR  categories  that  we  collected  categorized  the  Army  programs  by  the 
following: 


•  current  estimate, 

•  contractor, 

•  unit  cost  by  percentage  change, 

•  sunk  funding, 

•  cost  variance,  and 

•  schedule. 

The  sample  data  had  17  programs.  There  was  no  Army  SAR  data  available  for 
calendar  year  (CY)  2008.  We  then  placed  the  data  into  a  pivot  table  and  gave  the  data 
sparklines  to  identify  and  illustrate  trends.  The  dependent  variable  was  the  report 
categories’  cost  or  given  numerical  value.  The  independent  variable  was  the  SAR  report 
calendar  date. 

G.  CONCLUSION 

The  interviews  and  cost  data  were  used  to  gain  a  qualitative  and  quantitative 
perspective  on  the  topic.  The  interviews  provided  a  focused  view  of  the  legislation’s 
impact  on  a  program  office.  The  cost  data  provided  a  global  view  on  the  legislation’s 
effect  on  the  Army’s  MDAP  portfolio. 
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IV.  DATA  AND  ANALYSIS 


A,  INTRODUCTION 

This  chapter  addresses  the  organizational  structure  of  PEO  CS&CSS  followed  by 
an  analysis  of  interview  questions  driven  by  the  primary  and  subordinate  researeh 
questions  for  this  project.  These  questions  are  listed  below: 

1.  Overarehing  projeet  question:  How  are  WSARA  Title  II  policies  being 
applied  in  MDAPs? 

2.  Why  was  the  WSARA  needed,  or  what  were  the  precursors  that  led  to  the 
WSARA? 

3.  What  was  the  intended  effeet  of  the  WSARA? 

4.  How  effeetive  has  the  implementation  of  the  WSARA  been  in  PEO  CS&CSS? 

5.  Are  there  gaps  between  the  intent  and  application  of  Title  II  WSARA  within  a 
major  PEO,  why? 

The  intent  is  to  determine  whether  acquisition  reforms,  such  as  those  ineluded  in 
Title  II  of  the  WSARA  of  2009,  have  their  intended  effect  in  major  acquisition  programs. 
On-site  interviews  were  conducted  to  determine  the  extent  that  the  WSARA  is 
understood  and  implemented  in  PEO  CS&CSS.  Additional  data  were  eollected  through 
analysis  of  program  data  obtained  from  the  DAMIR  website  and  other  related  program 
doeuments  for  amplifying  information.  The  interview  questions  are  grouped  under  the 
thesis  questions,  followed  by  our  analysis  of  responses.  We  omit  the  first  question  in  this 
chapter  but  return  to  it  in  the  conclusions  and  recommendations  ehapter. 

B,  PEO  CS&CSS  ORGANIZATIONAL  STRUCTURE 

Major  aequisition  organizations  follow  a  structure  where  the  Program  Executive 
Offieer  (PEO)  and  his  or  her  deputies  orehestrate  the  operations  of  the  Program  Managers 
(PM,  civilian  or  military  member  in  the  grade  of  06  [colonel]  or  civilian  equivalent). 
Product  Managers  (PdM,  civilian  or  military  member  in  grade  05  [lieutenant  eolonel]  or 
civilian  equivalent),  with  their  various  assistants  and  staff  following  this  structure  as 
depicted  in  Eigure  8. 
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Figure  8.  Example  PEO  Staff  Strueture 


For  PEO  CS&CSS  the  PM  level  is  broken  down  into  the  following  categories: 

•  Force  Protection:  This  includes  bridging,  combat  engineering,  force 
sustainment,  petroleum  and  water,  sets,  kits,  outfits  and  tools  (SKOT),  test 
measurement  and  diagnostic  equipment  and  contingency  basing 
infrastructure. 

•  Joint  Program  Office  for  the  Joint  Light  Tactical  Vehicle  (JLTV): 

JLTV  is  the  joint  Army-Marine  acquisition  effort  currently  in  the 
Engineering  and  Manufacturing  Development  (EMD)  phase  of  the 
acquisition  life  cycle. 

•  Mine  Resistant  Ambush  Protected  Vehicles  (MRAP):  This  includes 
MRAP-ATV  (MATV)  systems  and  joint  logistics. 

•  Mobile  Electric  Power:  This  category  includes  small,  medium,  large,  and 
battery-powered  electrical  power  sources. 

•  Transportation  Systems:  This  category  is  comprised  of  personnel  and 
logistics  transportation  systems  including  medium  tactical  vehicles, 
armored  security  vehicles,  heavy  tactical  vehicles,  allied  tactical  vehicles 
and  Army  watercraft  systems. 

PEO  CS&CSS  manages  over  350  systems  in  total.  Our  initial  interviews  were 
conducted  with  various  representatives  within  this  organizational  structure. 

1,  Task  Allocation 

WSARA  legislation  called  for  additional  high-level  oversight  at  OSD.  The 
interviews  indicated  that  the  PEO  staff  structure  was  in  place  to  match  what  the  WSARA 
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required  at  OSD  level  as  shown  in  Figure  5  of  our  literature  review  seetion.  Projeet  and 
produet  offiees  had  teams  or  individuals  that  would  handle  data  ealls  required  by  PEO 
staff  or  OSD  level.  Additionally,  while  the  organization  was  struetured  in  aeeordanee 
with  regulation,  PdMs  were  eapable  of  adapting  their  strueture  based  on  their  program’s 
eurrent  phase  within  the  Integrated  Defense  Aequisition,  Teehnology,  and  Logisties  Life 
Cyele  Management  System. 

Work  tasks  are  alloeated  in  a  hierarehieal  fashion  from  PEO  to  projeet  offiee  to 
produet  offiee.  At  the  produet  offiee  level,  tasks  are  alloeated  by  PdM  identified 
priorities. 

C.  WSARA  QUANTITATIVE  IMPACTS 

The  intended  effeet  of  the  WSARA  was  “to  improve  the  organization  and 
proeedures  of  the  Department  of  Defense  for  the  aequisition  of  major  weapon  systems, 
and  for  other  purposes”  (WSARA  of  2009,  2009),  with  the  seeondary  intent  to  reduee 
risk  and  eosts.  To  aeeomplish  this,  the  legislation  foeused  on  ehanging  senior 
management  oversight  and  improving  early  deeision  variables  in  the  aequisition  system. 

As  established  in  the  literature  review,  the  need  for  the  law  was  elear.  Cost 
overruns  were  ever  inereasing  in  frequeney  and  value.  There  was  a  reeognized  need  “to 
improve  the  way  weapon  systems  are  aequired”  (Sullivan,  2012,  p.  4).  The  intent  of  the 
law  was  to  limit  those  overruns  and  enhanee  oversight  and  aeeountability  by  adding 
additional  layers  of  oversight  and  streamlining  the  monitoring  proeess  to  “ensure  that 
eosts  are  eontrolled”  (Sullivan,  2012,  p.  8). 

1.  DEFENSE  ACQUISITION  MANAGEMENT  INFORMATION 
RETRIEVAL 

DAMIR  eompiles  multiple  sourees  from  MDAP  and  Major  Automated 
Information  System  (MAIS)  information  systems.  Aeeording  to  its  website,  DAMIR  is 
“the  authoritative  souree  for  Seleeted  Aequisition  Reports  (SAR),  SAR  Baseline, 
Aequisition  Program  Baselines  (APB),  and  Assessments”  (DAMIR,  2013).  We  pulled 
relevant  data  from  DAMIR.  The  data  retrieved  were  from  the  Army’s  2005-2012  SARs 


27 


for  noted  MDAPs.  These  sourees  of  data  are  relevant  as  stated  by  the  Department  of 
Defense: 

SARs  summarize  the  latest  estimates  of  eost,  sehedule  and  performanee 
status.  These  reports  are  prepared  annually  in  eonjunetion  with  the 
submission  of  the  President’s  Budget.  Subsequent  quarterly  exeeption 
reports  are  required  only  for  those  programs  experieneing  unit  cost 
increases  of  at  least  15  percent  or  schedule  delays  of  at  least  six  months. 

The  total  program  cost  estimates  provided  in  the  SARs  include  research 
and  development,  procurement,  military  construction,  and  acquisition- 
related  operation  and  maintenance.  Total  program  costs  reflect  actual 
costs  to  date  as  well  as  future  anticipated  costs.  All  estimates  are  shown  in 
fully  inflated  then-year  dollars.  (DOD,  2013a) 

The  reports  were  chosen  because  they  reflect  the  most  relevant  data  on  the 
Army’s  MDAPs  from  2005  to  2012.  Those  years  were  selected  to  analyze  changes 
occurring  in  the  period  four  years  before  the  WSARA  and  three  years  after.  The  intent  of 
analyzing  the  years  before  and  after  the  WSARA  was  implemented  to  see  any  changes 
within  the  sample  on  cost  (which  was  the  WSARA’s  secondary  objective  and  also  the 
best  quantitative  measurement  for  success).  There  were  no  Army  SARs  listed  in  DAMIR 
for  CY2008. 

2,  Trends  in  Cost  Data 

We  specifically  analyzed  three  quantitative  cost  measures  to  see  whether  there 
were  any  trends:  current  estimate  total  cost;  Program  Acquisition  Unit  Cost  (PAUC) 
percent  change  compared  to  the  baseline  estimate;  and  Average  Procurement  Unit  Cost 
(APUC)  percent  change  compared  to  the  baseline  estimate.  We  were  looking  to  see 
whether  costs  were  increasing,  decreasing,  or  varied  before  and  after  implementation  of 
the  WSARA.  In  addition,  we  wanted  to  see  whether  there  was  variation  of  cost  changes 
before  and  after  implementation  of  the  WSARA.  Below  are  figures  from  our  quantitative 
analysis  that  show  the  trends  in  available  data  for  the  programs  based  off  the  current 
estimate,  PAUC,  and  APUC  for  each  program: 
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TRENDLINE 

PROGRAM 

2005 

2006 

2007 

2009 

2010 

2011 

2012 

1  ^ 

1 

1  N.  / 

1 

AH-64E  New 

Build 

2352.7 

2155.8 

2484.5 

AH-64E 

Remanufacture 

8990.4 

8996.4 

11686 

11893 

11968 

13760 

AMFJTRS 

9067.1 

9005.7 

4229.5 

3759.9 

CH-47F 

12798 

13449 

13350 

13636 

14439 

14257 

14387 

Excalibur 

2292.7 

2302.8 

2465.1 

2469.6 

1710.6 

1676.7 

1697.8 

'^-1-  -  — 

GMLRS/GMLRS 

AW 

16035 

6772.5 

6008.3 

6058.9 

6024.4 

6411.5 

6693.9 

lAMD 

5791.6 

6320.4 

6694.4 

6375.2 

JLENS 

7340.9 

7367.3 

7500.3 

8063.8 

8537.9 

2620.5 

2645.8 

I— ^ 

JTN 

1786.6 

2104.1 

1961.8 

1939 

1988.4 

1992.6 

2084.3 

JTRS  HMS 

11599 

11789 

3366.9 

5240.4 

5811.4 

9304.7 

10191 

MQ-IC  Gray 
Eagle 

5220.8 

5262.5 

4745.3 

4888.9 

PAC-3 

8527.7 

8521.7 

8524.9 

9326.1 

10002 

10797 

11007 

•  ’IV— 

Patriot/M  EADS 
CAP 

30531 

30178 

29895 

30827 

12563 

12952 

12832 

1  / 

PIM 

7899.3 

7904.2 

UH-60M  Black 

Hawk 

23770 

23830 

24043 

23682 

27341 

28861 

25694 

WIN-T  Inc  2 

3870.8 

4997.7 

6352.5 

6461.3 

5137.4 

WIN-T  Inc  3 

15964 

16056 

14456 

17890 

, 

Grand  Total 

114680 

115304 

109983 

153970 

145660 

147482 

149433 

Figure  9.  SAR  Current  Estimate,  Total  Cost  in  Millions  ($) 
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TRENOUNE 

PROGRAM 

2005 

2006 

2007 

2009 

2010 

2011 

2012 

1  N. 

• 

AH*64ENew  Build 

-0.74% 

-13.78% 

-8.11% 

1 

AH-$4E  Remanufacture 

2.33% 

2.92% 

28.41% 

-0.15% 

-2.09% 

10.53% 

1*- 

1  \ 

1  X 

AMF  JTRS 

2.45% 

1.05% 

6.99% 

-27.62% 

CH-47F 

3.18% 

8.59% 

8.28% 

10.73% 

15.17% 

13.10% 

13.48% 

Excalibur 

•4.62% 

-4.62% 

1.39% 

4.17% 

1.81% 

•0.90% 

-4.07% 

- 

GMLRS/GMLRS  AW 

17.14% 

68.57% 

2.86% 

6.67% 

5.71% 

9.52% 

12.38% 

lAMD 

0.00% 

11.09% 

-26.33% 

-26.58% 

• - - - - r - ' 

JLENS 

2.20% 

1.36% 

4.08% 

12.35% 

17.88% 

215.72% 

217.46% 

1 

1 

*  *  ■ - m  ,  . 

1 

JTN 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

JTRSHMS 

3.85% 

3.85% 

7.69% 

-26.92% 

-19.23% 

0.00% 

3.33% 

MQ-IC  Gray  Eagle 

0.00% 

-4.38% 

-14.53% 

-12.74% 

PAC-3 

25.03% 

23.47% 

23.64% 

16.99% 

13.31% 

12.67% 

9.37% 

Patriot/M  EADS  CAP 

-5.37% 

-7.76% 

-7.47% 

3.97% 

8.94% 

10.44% 

9.11% 

I  \ 

PIM 

0.00% 

-1.47% 

UH-60M  Black  Hawk 

9.61% 

9.08% 

12.69% 

16.30% 

18.07% 

22.20% 

8.69% 

WIN-T  Inc  2 

0.00% 

10.68% 

8.32% 

8.63% 

13.40% 

1^ 

WIN-T  Inc  3 

-8.77% 

-8.00% 

-14.10% 

-11.26% 

Figure  10.  SAR  PAUC  %  Change  Against  Original  Baseline 
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SAR  APUC  %  Change  Against  Originai  Estimate  Baseline 


TRENDUNE 

PROGRAM 

2005 

2006 

2007 

2009 

2010 

2011 

2012 

AH-64E  New  Build 

0.00% 

0.00% 

0.00% 

0.00% 

-0.74% 

-13.78% 

-8.11% 

AH-64E  Remanufacture 

0.00% 

4.07% 

4.65% 

36.49% 

-0.01% 

-2.46% 

13.74% 

* — — ' — [* — 

AMF  JTRS 

0.00% 

0.00% 

0.00% 

1.36% 

0.00% 

-23.18% 

-53.64% 

1 

CH-47F 

3.20% 

8.70% 

8.38% 

10.91% 

15.45% 

13.34% 

13.72% 

Excalibur 

0.00% 

0.00% 

15.38% 

20.51% 

5.26% 

0.00% 

-5.26% 

GMLRS/GMLRSAW 

19.70% 

59.09% 

3.30% 

4.40% 

4.40% 

7.69% 

10.99% 

— 

1  \ 

1  ^ 

lAMD 

0.00% 

0.00% 

0.00% 

0.00% 

2.20% 

■30.80% 

■37.76% 

JLEN5 

2.15% 

1.22% 

5.17% 

11.53% 

13.31% 

0.00% 

0.00% 

- 1 - 

1 

1 

JTN 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

JTRS  HMS 

0.00% 

0.00% 

-20.00% 

-36.00% 

-36.00% 

-7.14% 

-3.57% 

MQ  IC  Gray  Eagle 

0.00% 

0.00% 

0.00% 

0.00% 

-1.62% 

-8.22% 

-6.18% 

PAC-3 

25.35% 

24.20% 

24.52% 

23.74% 

24.11% 

27.49% 

24.74% 

Patriot/M  EADS  CAP 

-5.29% 

-6.66% 

-6.46% 

3.84% 

6.23% 

7.03% 

5.68% 

-t-  \ 

PIM 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

-1.67% 

UH-60M  Black  Hawk 

9.70% 

9.21% 

12.88% 

16.48% 

19.02% 

23.31% 

9.22% 

i — ^ 

WIN-Tlnc2 

0.00% 

0.00% 

0.00% 

11.13% 

9.39% 

9.50% 

13.68% 

WIN-Tlnc3 

0.00% 

0.00% 

0.00% 

-7.97% 

-7.45% 

-15.92% 

-10.82% 

Figure  1 1 .  SAR  APUC  %  Change  Against  Original  Baseline 


D.  CENTRAL  TENDENCY 

To  understand  variation  in  cost  measure,  we  analyzed  three  measures  of 
variability  in  central  tendency.  The  intent  was  to  find  the  measurement  that  provides  the 
best  representation  of  the  data.  We  used  the  SARs  to  gather  data  on  the  standard 
deviations,  averages,  and  coefficient  of  variation  (CV)  for  the  specified  cost  element 
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current  estimate,  PAUC,  and  APUC  %  ehanges).  First,  averages  (means)  were  used  to 
gauge  the  total  average  of  eaeh  program.  Seeond,  standard  deviation  was  used  to  view 
the  average  deviation  of  values  from  the  mean.  Third,  CV  was  used  to  see  the  dispersion 
of  data  relative  to  the  mean  because  it  allows  for  analysis  of  variation  between  the 
programs  that  have  signifieant  eost  differenees. 

The  data  provided  by  DAMIR  had  17  Army  MDAP  programs  listed  from  2005  to 
2012.  Data  taken  as  “before  WSARA”  was  from  2005  to  2009  SARs.  Data  taken  as 
“after  WSARA”  was  from  2010  to  2012  SARs.  For  averages,  standard  deviation,  and 
coeffieient  of  variation,  the  data  were  taken  from  eaeh  program’s  SAR  before  and  after 
the  WSARA,  respeetively. 

Table  1  summarizes  the  data  eollected  and  calculated;  the  details  are  discussed  in 
the  following  three  paragraphs.  Dollar  values  are  in  millions.  The  pereent  ehanges  for 
PAUC  and  APUC  are  the  ehange  eompared  to  the  original  baseline  estimate  from  the 
SAR. 


Table  1 .  SAR  Measures  of  Central  Tendeney 


AVERAGES 

Current 

PAUC% 

APUC% 

Estimate 

Change 

Change 

Programs  that  increased 

11 

5 

5 

Average  increase  before  WSARA 

$10,210.16 

5.16% 

8.69% 

Average  increase  after  WSARA 

$10,370.26 

40.04% 

14.79% 

Average  increase  net 

$160.09 

34.88% 

6.10% 

Programs  that  decreased 

6 

12 

12 

Average  decrease  before  WSARA 

$10,552.49 

4.72% 

3.13% 

Average  decrease  after  WSARA 

$6,014.36 

-2.73% 

-6.02% 

Average  decrease  net 

$(4,538.13) 

-lASVo 

-9.15% 

All  Programs  average  before 
WSARA 

$10,347.09 

4.87% 

4.98% 

All  Programs  average  after 

WSARA 

$8,832.88 

9.85% 

0.10% 

All  Programs  average  net  change 

$(1,514.21) 

4.98% 

-4.06% 

STANDARD  DEVIATIONS 

Current 

PAUC% 

APUC% 

Estimate 

Change 

Change 

Programs  that  Increased 

9 

8 

10 

stdev  increase  before  WSARA 

$297 

0.0418 

0.0308 

stdev  increase  after  WSARA 

$1,198 

0.2221 

0.0979 

stdev  increase  net 

$901 

0.1803 

0.0671 

Programs  that  decreased 

8 

9 

7 

stdev  decrease  before  WSARA 

$1,572 

0.0953 

0.0963 

stdev  decrease  after  WSARA 

$601 

0.0357 

0.0308 

stdev  decrease  net 

$(971) 

(0.0596) 

(0.0655) 

All  Programs  stdev  before  WSARA 

$1,224 

0.0856 

0.0578 

All  Programs  stdev  after  WSARA 

$917 

0.1234 

0.0702 

All  Programs  stdev  net  change _ $(307) _ 0.0379 _ 0.0125 


COEFFICIENT  OF  VARIATION 


Current 

PAUC% 

APUC% 

Estimate 

Change 

Change 

Programs  that  increased 

10 

1 

10 

CV  increase  before  WSARA 

2.76% 

58.92% 

68.29% 

CV  increase  after  WSARA 

16.40% 

117.74% 

101.69% 

CV  increase  net 

13.64% 

58.82% 

33.40% 

Programs  that  decreased 

7 

7 

5 

CV  decrease  before  WSARA 

22.50% 

226.34% 

124.99% 

CV  decrease  after  WSARA 

8.25% 

95.22% 

39.65% 

CV  decrease  net 

-14.24% 

-131.12% 

-85.34% 

All  Programs  CV  before  WSARA 

15.32% 

176.11% 

99.79% 

All  Programs  CV  after  WSARA 

13.04% 

106.48% 

81.01% 

All  Programs  CV  net  change 

-2.27% 

-69.63% 

-18.78% 

1.  SAR  Current  Estimate 

The  first  cost  estimate  was  the  SAR  “current  estimate,”  which  is  the  most  recent 
estimate  of  the  program’s  parameters  reflecting  the  President’s  Budget  proposal  as  well 
as  unforeseen  circumstances  or  unavoidable  circumstances  that  adjusted  cost  measures. 
(Hagan,  2012,  p.  B-55). 
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Before  WSARA  implementation,  1 1  programs  inereased  in  dollar  value  from  the 
eurrent  estimate.  The  average  value  of  the  11  programs  before  the  WSARA  was  $10.21 
billion.  The  average  cost  estimate  of  the  11  programs  after  the  WSARA  was  $10.37 
billion,  yielding  a  net  increase  of  $160  million.  After  WSARA  implementation,  six 
programs  decreased  in  dollar  value  from  the  current  estimate.  The  average  value  of  the 
six  programs  before  the  WSARA  was  $10.55  billion.  The  average  value  of  the  six 
programs  after  the  WSARA  was  $6.01  billion,  yielding  a  decrease  of  $4.54  billion.  The 
average  of  all  17  programs  before  the  WSARA  was  $10.35  billion.  After  the  WSARA 
the  average  was  $8.83  billion,  yielding  a  net  decrease  of  $1.51  billion. 

The  11  programs  that  increased  only  increased  a  very  small  amount.  The  6 
programs  that  decreased  nearly  3  times  over  the  average  decrease  of  the  entire  Army 
portfolio.  However,  this  could  only  be  an  indicator  or  program  cancellations,  or 
appropriations  funding  being  cut.  Therefore,  the  dollar  value  is  not  a  good  indicator 
because  of  funding  types  and  varying  prices  of  programs  within  the  portfolio. 

From  before  and  after  WSARA  implementation,  the  standard  deviation  of  nine 
programs  increased  from  $297.38  million  to  $1,198.02  million.  Eight  programs’  standard 
deviations  decreased  from  $1,572  million  to  $601  million.  The  combined  standard 
deviation  difference  between  programs  before  and  after  WSARA  implementation 
indicated  a  $307  million  net  decrease. 

The  standard  deviation  is  a  slightly  better  metric  for  interpreting  current  estimate 
changes.  The  programs  that  increased  did  so  4  times  in  magnitude,  indicating  that 
programs  that  did  increase  in  standard  deviation  are  becoming  more  spread  out  from  the 
average.  The  programs  that  decreased  did  so  by  2.6  times  in  magnitude,  showing  a 
slightly  less  dispersion  around  the  mean.  When  comparing  all  programs,  they  indicate 
less  dispersion  but  only  a  slight  decrease. 

From  before  and  after  WSARA  implementation,  the  CV  of  10  programs  increased 
from  2.76%  to  16.40%.  Seven  programs’  CVs  decreased  from  22.5%  to  8.25%  after  the 
WSARA  was  implemented;  a  decrease  of  14.25%.  The  average  of  all  programs’  CV 
went  from  15.32%  before  to  13.04%  after  the  WSARA,  yielding  a  decrease  of  2.27%. 
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As  stated  before,  CV  shows  the  dispersion  of  data  relative  to  the  mean  beeause  it 
allows  for  analysis  of  variation  while  eliminating  the  signifieant  eost  differences  and 
providing  the  answer  in  percent  form.  The  current  estimate  overall  decrease  is  not  very 
significant.  The  program  implication  is  that  current  estimates  have  become  more  in 
control  by  2.27%. 

2.  PAUC  Percent  Change 

PAUC  is  the  program  acquisition  unit  cost.  It  is  computed  by  “dividing  the 
Program  Acquisition  Cost  by  the  Program  Acquisition  Quantity”  (Hagan,  2012,  p.  B- 
177).  It  is  basically  how  much  everything  costs  divided  by  the  amount  of  units  that  are 
(going  to)  be  purchased.  We  compared  the  PAUC  percent  change  from  the  original 
baseline  estimate  per  each  program  SAR. 

From  before  and  after  WSARA  implementation,  five  programs’  PAUC  increased 
from  5.16%  to  40.04%,  a  net  increase  of  34.88%.  12  programs’  PAUC  percentage 

decreased  from  4.72%  to  -2.73%,  a  net  decrease  of  -7.45%.  The  combined  average  for 
all  17  programs’  PAUC  percentage  change  against  the  baseline  estimate  increased  from 
4.87%  before  the  WSARA  to  9.85%  after,  a  net  increase  of  4.98%. 

From  before  and  after  WSARA  implementation,  the  standard  deviation  of  eight 
programs  increased  from  0.0418  before  to  0.2221  after,  a  net  increase  of  0.1803.  Nine 
programs  decreased  from  0.0953  before  to  0.0357,  a  net  decrease  of  0.0596.  The 
combined  standard  deviation  average  increased  from  0.0856  to  0.1234,  a  net  increase  of 
0.0379. 

From  before  and  after  WSARA  implementation,  the  CV  of  seven  programs 
increased  from  58.92%  to  117.74%,  an  increase  of  58.82%.  Seven  programs  decreased 
from  226.34%  to  95.22%,  a  decrease  of  131.12%.  Two  programs  did  not  have  data  to 
allow  CV  calculation  before  the  WSARA.  The  combined  average  for  14  programs  with 
available  data  to  calculate  CV  yielded  a  176.11%  before  to  106.48%  after,  a  net  decrease 
of  69.63. 
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The  programs  PAUC  that  did  only  increase  did  so  by  34.88%.  When  compared  to 
the  standard  deviation  (.1803)  of  increasing  programs,  it  a  clear  indicator  of  programs 
becoming  more  expensive.  However,  although  this  is  highlighted  with  the  CV  of 
58.82%,  the  programs  that  decreased  did  so  significantly  by  131.12%. 

3.  APUC  Percent  Change 

APUC  is  average  procurement  unit  cost.  It  is  “calculated  by  dividing  total 
procurement  cost  by  the  number  of  articles  to  be  procured”  (that  is,  recurring  and 
nonrecurring  costs  associated  with  production  of  the  item  (Hagan,  2012,  p.  B-19).  It  is 
basically  the  costs  associated  with  physically  making  the  product  or  item.  We  compared 
the  APUC  percent  change  from  the  original  baseline  estimate  per  each  program  SAR. 

From  before  and  after  WSARA  implementation,  five  programs  increased  from 
8.69%  to  14.79%,  an  increase  of  6.1%.  12  programs  decreased  from  3.13%  to 
6.02%,  a  decrease  of  9.15%.  The  combined  average  change  in  APUC  decreased  from 
4.98%  before  to  0.10%  after,  a  net  decrease  of  4.06%. 

From  before  and  after  WSARA  implementation,  10  programs’  standard  deviations 
increased  from  0.0308  to  0.0979,  an  increase  of  0.0671.  Seven  programs  decreased  from 
0.0963  to  0.0308,  a  decrease  of  0.0655.  The  combined  APUC  standard  deviation 
increased  from  0.0578  to  0.0702,  a  net  increase  of  0.0125. 

From  before  and  after  WSARA  implementation,  the  CV  of  10  programs  increased 
from  68.29%  to  101.69%,  an  increase  of  33.40%.  Five  programs  decreased  from 
124.99%  to  39.65%,  a  decrease  of  85.34%.  The  combined  average  CV  decreased  from 
99.79%  to  81.01%,  a  net  decrease  of  18.78%.  Two  programs  did  not  have  data  to  allow 
CV  calculation  before  the  WSARA. 

Average  procurement  unit  costs  are  decreasing  on  average.  The  standard 
deviation  increase  shows  that  they  are  becoming  more  difficult  to  control;  however,  the 
CV  indicates  that  the  APUC  for  the  Army  portfolio  is  decreasing. 
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E,  INTENDED  EFFECTS  OF  THE  WSARA 

This  section  is  based  off  the  interviews  we  conducted.  Each  subsection  of  the 
WSARA’s  Title  II  focused  on  changes  to  current  or  new  acquisition  policy.  These 
questions  were  part  of  the  interview  process  conducted  at  PEO  CS&CSS  to  determine 
whether  and  how  these  legislative  requirements  are  implemented  within  this  organization. 
Interview  questions  were  primarily  based  on  sections  of  WSARA  Title  II. 

1.  Consideration  of  Trade-offs  for  Requirement  Development 

Section  201  of  the  WSARA  states  that  “the  Secretary  of  Defense  shall  ensure  that 
mechanisms  are  developed  and  implemented  to  require  consideration  of  trade-offs  among 
cost,  schedule,  and  performance  objectives  as  part  of  the  process  for  developing 
requirements  for  Department  of  Defense  acquisition  programs”  with  the  duty  of  the 
Milestone  Decision  Authority  to  ensure  that  “the  program  is  affordable”  (Weapon  System 
Acquisition  Reform  Act  of  2009,  2009). 

Program  requirements  are  generated  at  the  Army  G3/G4  level  and  disseminated 
to  PEOs  and  PMs.  PMs  have  little  say  on  MDAPs  to  control  trade-offs  among  cost, 
schedule,  and  performance  objectives.  However,  the  project  managers  did  have  influence 
on  the  MDA  decision-making  process  on  trade-offs  as  indicated  by  WSARA  201(f) 
DUTIES  OE  MILESTONE  DECISION  AUTHORITY; 

Section  2366b(a)(l)(B)  of  such  title  is  amended  by  inserting  “appropriate 

trade-offs  among  cost,  schedule,  and  performance  objectives  have  been 

made  to  ensure  that”  before  “the  program  is  affordable.” 

Trade-offs  had  typically  been  made  by  the  time  the  current  project  manager  was 
selected  and  was  actively  managing  a  project.  Objectives  are  typically  already  set  in  the 
program  baseline  CPD.  If  there  are  changes,  project  managers  must  provide  rationale  for 
the  need  to  change  their  baseline  estimates  in  cost,  schedule,  or  performance. 

Certain  programs  had  less  stringent  trade-off  requirements.  Eor  example,  the 
MRAP  program  could  change  baseline  estimates  to  meet  dynamic  requirements  driven  by 
the  Global  War  on  Terror  situations  in  Iraq  and  Afghanistan.  As  one  PM  put  it,  “Cost 
was  third,  schedule  was  second,  and  actually  you  balance  it  based  on  theater  priorities.” 
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The  Joint  Light  Tactical  Vehicle  (JLTV)  was  a  cost  informed  trade  driven  platform. 
Tradeoff  requirements  were  pushed  to  the  contractor  level  based  off  of  tiered  capability 
development  documents  (CDDs)  and  KPPs. 

The  spirit  of  the  legislation  matches  what  the  PMs  were  doing  with  their 
respective  projects,  depending  on  state  of  the  acquisition  cycle.  WSARA  Section 
201(a)(2)(A-B)  states  that: 

(A)  Department  of  Defense  officials  responsible  for  acquisition,  budget, 
and  cost  estimating  functions  are  provided  an  appropriate  opportunity  to 
develop  estimates  and  raise  cost  and  schedule  matters  before  performance 
objectives  are  established  for  capabilities  for  which  the  Chair-man  of  the 
Joint  Requirements  Oversight  Council  is  the  validation  authority;  and  (B) 
the  process  for  developing  requirements  is  structured  to  enable 
incremental,  evolutionary,  or  spiral  acquisition  approaches,  including  the 
deferral  of  technologies  that  are  not  yet  mature  and  capabilities  that  are 
likely  to  significantly  increase  costs  or  delay  production  until  later 
increments  or  spirals. 

There  was  clear  evidence  that  the  JLTV  program  did  exactly  what  WSARA 
201(c)  required  of  review  of  joint  military  requirements.  However,  the  older  programs 
that  are  were  already  established  did  not  meet  this  requirement. 

2.  Competition  Policy 

The  WSARA  mandates  competition  in  as  many  phases  of  acquisition  as  possible. 
WSARA  section  202  makes  suggestions  for  competition  during: 

(1)  Competitive  prototyping; 

(2)  Dual-sourcing; 

(3)  Unbundling  of  contracts; 

(4)  Funding  of  next-generation  prototype  systems  or  subsystems; 

(5)  Use  of  modular,  open  architectures  to  enable  competition  for  upgrades; 

(6)  Use  of  build-to-print  approaches  to  enable  production  through  multiple 
sources; 

(7)  Acquisition  of  complete  technical  data  packages; 

(8)  Periodic  competitions  for  subsystem  upgrades; 

(9)  Licensing  of  additional  suppliers; 
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(10)  Periodic  system  or  program  reviews  to  address  long-term  competitive 
effects  of  program  decisions  (Weapon  System  Acquisition  Reform  Act  of 
2009,  2009). 

PEO  CS&CSS  competition  policies  were  assessed  to  determine  the  strategies 
employed  to  ensure  competition  of  products  given  the  stage  of  the  product  in  the 
acquisition  life  cycle.  Reverse  engineering  techniques  are  the  primary  methods  employed 
to  create  their  own  technical  data  package  (TDP),  which  would  drive  more  competition, 
because  the  contractors  put  a  high  cost  on  the  procurement  of  TDPs.  In  addition,  the  PMs 
primarily  used  sole-sourcing  due  to  priorities  and  the  “finite  amount  of  OEMs  [original 
equipment  manufacturers]”  in  the  given  industry  or  program. 

In  contrast,  the  MRAP  and  JETV  program  focused  on  values  trade-offs  with 
performance-based  competitions  that  helped  industry  partners  be  successful,  as  well  as 
maintaining  competition  requirements.  The  need  to  develop  the  program  requirements 
and  have  competition  before  Milestone  A  with  defense  acquisition  executive  (DAE)  and 
component  acquisition  executive  (CAE)  support,  would  facilitate  enforcement. 

3,  Prototyping 

WSARA  section  203  requires  prototyping  for  MDAPs  before  Milestone  B 
approval.  PEO  CS&CSS  policy  concerning  prototyping,  its  frequency  of  use,  potential 
costs,  and  the  number  of  times  prototyping  is  waived  was  analyzed  to  determine  the  level 
of  compliance  with  this  WSARA  requirement. 

WSARA  203:  (a)  COMPETITIVE  PROTOTYPING.— Not  later  than  90 
days  after  the  date  of  the  enactment  of  this  Act,  the  Secretary  of  Defense 
shall  modify  the  guidance  of  the  Department  of  Defense  relating  to  the 
operation  of  the  acquisition  system  with  respect  to  competitive 
prototyping  for  major  defense  acquisition  programs 

Prototyping  is  required  unless  the  requirement  is  waived  by  the  MDA,  prototypes 
exceed  the  expected  life-cycle  benefits,  or  DOD  is  unable  to  meet  critical  national 
security  objectives. 

We  assessed  that  prototyping  does  not  typically  happen  in  production  phases.  As 

most  of  the  programs  at  PEO  CS&CSS  were  in  the  later  phases,  the  need  to  prototype 

was  not  as  prevalent.  However,  there  were  some  cost  savings  incentives  to  prototype  if 
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modifications  to  systems  were  being  made.  In  that  ease,  the  PMs  typieally  used  RDEC 
(Researeh,  Development  and  Engineering  Center)  to  develop  a  prototype  or  a  TDP  and 
then  had  eontraetors  develop  their  solutions  for  the  solieitation,  thus  improving 
eompetition. 

In  eontrast,  rapid  initiatives  (like  MRAP)  did  not  have  prototypes  as  per  the 
WSARA  requirement.  The  urgent  needs  of  the  Army  drove  the  requirement,  and  they 
had  eompanies  produee  units  based  off  performanee  specs  to  get  equipment  out  to  the 
troops  faster  and  leverage  the  waiver  elements  of  the  legislation. 

4,  Systemic  Problems  and  Product  Termination 

Eor  this  section,  our  research  efforts  focused  on  the  policies  and  procedures 
employed  by  PEO  CS&CSS  concerning  identification  of  systemic  problems  and  methods 
of  determining  when  a  product  line  must  be  terminated  to  determine  if  deviations  from 
WSARA  policy  intent  and  application  exist. 

The  systemic  problems  were  typically  identified  through  internal  reporting 
procedures  at  project  level  and  presented  to  the  MDA.  If  problems  are  identified, 
WSARA  Title  II  sec  204  requires  the  MDA  to  certify  costs,  needs,  program  duplicates, 
relevancy,  and  that  depot-level  maintenance  and  repair  capabilities  have  been  made,  and 
conduct  an  analysis  of  alternatives  consistent  with  DC  APE  guidance. 

WSARA  sec  206  addresses  product  termination.  The  PMs  reported  changes  to 
cost,  performance,  or  schedule.  If  the  changes  were  detrimental,  the  WSARA  requires 
the  PM  to  determine  the  root  cause  of  cost  growth  or  schedule  threshold  breach.  The 
reports  for  these  changes  followed  the  same  reporting  processes  as  the  systemic 
problems.  However,  it  is  the  MDA  that  ultimately  determines  whether  a  product  or 
project  is  continued  or  terminated. 

The  reporting  structure  in  PEO  CS&CSS  allowed  for  multiple  reports  on  cost 
performance  and  schedule  from  product  managers  up  to  the  project  managers  to  the  PEO. 
This  facilitated  the  requirement  from  WSARA  204  with  each  product/project  manager 
able  to  customize  their  reports  to  explain  an  issue  at  hand.  Recommendations  made  at  the 
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PM  level  with  supporting  cost,  schedule,  and  performance  data  facilitated  decision¬ 
making  capabilities  for  the  MDA  on  issues  and  termination  if  needed. 


5.  Effectiveness  of  WSARA  Policies 

WSARA  policy  relevance  and  cost  effectiveness  for  PEO  CS&CSS  was 
addressed  to  determine  whether  there  is  added  value  from  this  policy  within  the  PEO. 
Cost  effectiveness  is  a  measure  of  the  results  added  by  a  system  or  in  this  case,  legislative 
requirements  (Hagan,  2012,  p.  B-49).  We  determined  that  the  WSARA  legislation 
requiring  the  policies  that  PEO  CS&CSS  followed  were  generally  cost  effective. 
However,  there  was  some  distaste  for  the  reporting  requirements;  although  deemed 
necessary,  the  reports  may  not  provide  significant  value. 

F.  WSARA  IMPLEMENTATION 

Eor  this  section,  we  focused  our  inquiries  on  reporting  requirements  and  personnel 
evaluation  criteria  within  PEO  CS&CSS.  Studying  these  items  enhanced  our  abilities  to 
determine  what  is  important  to  the  PEO  and  whether  or  not  WSARA  policies  are  among 
the  list  of  important  items.  We  gauged  their  significance  by  level  and  frequency  of 
reporting  and  whether  or  not  WSARA  policy  items  are  included  in  personnel  evaluation 
criteria.  WSARA  assists  with  comparing  and  evaluating  the  importance  of  these 
reporting  policies  within  the  PEO  and  whether  or  not  they  appear  to  be  effective  and 
efficient. 

1.  Cost,  Schedule,  and  Performance  Reports 

The  GAO  study  mentioned  in  the  beginning  of  this  project  stated  that  incentives 
are  partially  to  blame  for  inadequacies  in  acquisitions  (Erancis,  2009).  We  wanted  to 
determine  how  incentives  influenced  the  reporting  process.  We  learned  that  acquisition 
managers  and  leaders  are  incentivized  to  maximize  fund  obligation  rates  more  than 
anything  else.  While  there  is  a  push  to  meet  the  needs  of  soldiers  and  that  is  typically 
where  acquisitions  begin,  somewhere  along  the  line,  it  becomes  about  obligating  funds. 

In  terms  of  what  is  reported,  the  research  indicated  a  wide  range  of  differences. 
Reform  efforts  (like  the  WSARA)  stimulate  additional  reporting.  Our  researched  indicated 
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that  there  are  more  reports  required  than  ean  be  numbered.  The  aequisition  proeess 
beeomes  muddied  when  reform  efforts  ereate  deviations  from  standard  reporting 
requirements.  Reporting  has  beeome  sueh  an  arduous  proeess  at  the  PEO  that  personnel 
must  be  alloeated  solely  to  traeking  reports  and  ensuring  that  reports  are  submitted  on  time. 
Mueh  of  this  reporting  appears  to  be  required  from  those  initiating  the  reform  efforts  so  that 
these  individuals  have  evidenee  that  their  reforms  are  ereating  ehange. 

Programs  at  PEO  CS&CSS  in  earlier  phases  of  the  aequisition  life  eyele  have 
fewer  reporting  requirements  than  those  in  the  latter  phases.  Some  of  the  eommon 
reports  inelude:  a  weekly  report  sent  to  everyone  in  their  produet  line,  weekly  staff  ealls, 
metries  and  portfolio  reviews,  and  Commander’s  Critieal  Information  Requirements 
(CCIR)  reporting. 

We  determined  that  PEO  CS&CSS  management  trust  that  all  of  their  reports  go  to 
a  higher  organization  beyond  their  PEO,  albeit  a  defense  aequisition  exeeutive  (DAE)  if 
one  exists.  Army  Budget  Offiee,  or  Congress.  In  terms  of  whieh  reports  are  more 
important  than  others,  there  appears  to  be  a  foeus  on  obligation  of  funds  and  that  these 
reports  are  prioritized  with  regard  to  what  is  submitted  to  higher  tiers. 

There  are  not  striet  reporting  struetures.  We  determined  that  program  eost 
reporting  is  stressed  above  all  else  when  we  studied  PEO  reporting  hierarehies. 
Espeeially  in  times  of  fiseal  eonstraints,  reporting  has  beeome  even  more  about  eost. 
However,  this  is  not  reporting  to  show  how  eosts  are  mitigated.  Rather,  this  reporting 
foeuses  on  obligation  rates.  There  seems  to  be  little  eoneern  with  holding  managers  to 
items  mentioned  within  WSARA  Title  II  polieies.  The  foeus  is  on  ensuring  funds  are 
obligated  fully  and  on  time.  None  mentioned  having  to  report  speoifieally  on  any  aspeet 
of  the  WSARA  reforms. 

2.  Reporting  Management 

Report  oversight  is  another  faetor  used  in  our  researeh  to  indieate  the  importanee 
of  what  is  reported.  Those  items  with  greatest  oversight  are  determined  to  be  most 
important  to  the  PEO  and  their  superiors.  Our  analysis  for  this  seetion  foeused  on  PEO 
CS&CSS  reporting  proeesses,  ineluding  the  number  of  reports,  and  the  level  of  oversight 
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or  senior  management  involvement.  The  level  of  oversight  varied  based  on  where  a 
respective  program  fell  within  the  acquisition  life  cycle.  Those  programs  still  in  the  early 
phases  of  the  acquisition  life  cycle  stated  that  they  feel  there  is  little  oversight  to  what 
they  do  on  a  regular  basis.  However,  managers  of  programs  operating  in  the  latter  phases 
of  the  acquisition  life  cycle  stated  they  are  “over  sighted  to  death.” 

While  oversight  varied  depending  on  acquisition  life  cycle  phase,  there  was  a 
consensus  that  reports  were  not  actually  reviewed  beyond  the  PEO  level  based  on  the 
absence  of  feedback.  Often,  feedback  is  received  concerning  reporting  only  if  there  is  a 
combat-related  requirement.  For  instance,  if  a  soldier  is  wounded  or  killed  in  combat 
while  employing  a  product  in  their  program,  this  generates  changes  to  an  aspect  of  their 
product  (armor  for  example),  and  oversight  becomes  abundant.  Otherwise,  for  programs 
in  early  acquisition  phases,  there  seems  to  be  little  oversight  to  what  is  going  on  in  a 
program.  When  a  problem  presents  itself  in  a  program  is  another  instance  of  when  report 
feedback  is  prominent.  Specifically,  reports  concerning  funding  problems  receive  an 
abundance  of  feedback  along  with  additional  oversight  indicating  that  funding  and  cost 
are  valued  above  all  other  report  items. 

3.  Performance  Assessments  and  the  WSARA 

Performance  evaluations  and  metrics  are  a  method  employed  by  supervisors  to 
incentivize  behavior  in  congruence  with  organizational  goals.  Investigating  whether 
personnel  evaluations,  within  PEO  CS&CSS,  included  requirements  from  the  WSARA 
demonstrates  the  level  of  the  WSARA  policies  significance  to  the  PEO.  We  focused  our 
research  on  specific  WSARA  aspects  including  reporting,  cost  control,  and  requirements 
development,  and  analyzed  whether  or  not  these  policies  were  evaluated  in  either  the 
civilian  or  military  performance  evaluations.  We  discovered  that  personnel  are  not 
evaluated  on  (nor  evaluate  their  subordinates  on)  anything  directly  stemming  from 
WSARA  policies.  Considering  adherence  to  policy  initiatives  when  conducting 
evaluations  is  something  worth  implementing  for  managers  and  those  under  their 
evaluation.  Evaluations  at  PEO  CS&CSS  are  more  about  percentage  of  funds  obligated 
or  number  of  upgrades  installed,  for  example,  than  they  are  about  managing  cost, 
schedule,  and  performance. 
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There  is  discontinuity  between  civilian  and  military  personnel  evaluations  as  well 
according  to  those  interviewed.  Civilian  evaluations  are  very  vague  and  do  not  tell  much 
about  their  performance  unless  they  did  something  wrong.  The  evaluation  process  is 
further  complicated  due  to  lacking  longevity  of  military  personnel  within  a  program 
office.  As  such,  evaluations  become  more  about  what  kind  of  person  the  military 
individual  is,  rather  than  how  they  manage  a  program  with  regard  to  items  such  as  those 
mentioned  in  the  WSARA  policies.  Because  that  individual  is  not  in  the  program  from 
conception  through  final  phases  of  product  acquisition,  it  is  difficult  to  measure  their 
cost,  performance,  and  schedule  contributions. 

4,  Reporting  Effectiveness 

We  assessed  the  effectiveness  of  the  PEO  CS&CSS  reporting  process  and 
contents  of  submitted  reports  to  identify  whether  inefficiencies  exist  and  ascertain  the 
implications  of  continued  acquisition  reforms  (such  as  the  WSARA)  on  this  process.  We 
established  that  the  current  reporting  process  is  not  effective.  The  consensus  was  that  the 
acquisition  process  itself  is  very  effective.  However,  everything  that  is  continually  added 
on  top  of  this  process,  such  as  WSARA  reforms  and  the  reporting  requirements  that  come 
with  reform,  take  away  from  the  process’s  effectiveness.  There  are  opportunity  costs 
associated  with  having  to  obligate  personnel  in  a  program  to  managing  nothing  but 
reporting  requirements.  This  takes  away  from  their  abilities  to  add  real  value  to  other 
aspects  within  a  program. 

Additionally,  there  is  little  feedback  with  regard  to  the  reports  that  they  submit  on 
a  regular  basis.  New  policies  often  create  new  reporting  requirements,  but  acquisition 
managers  are  not  confident  that  these  reports  amount  to  much  more  than  simply 
satisfying  a  requirement  to  report.  Policy  reforms  that  lead  to  additional  reporting 
requirements  appear  to  be  nothing  more  than  another  person’s  (the  person  implementing 
the  reform)  attempt  to  have  their  part  in  history.  This  leads  to  ineffective  reporting  due  to 
the  time  it  takes  to  generate  these  reports  and  the  tasks  that  may  fall  by  the  wayside 
because  time  was  spent  reporting  rather  than  managing  cost,  schedule,  performance  and 
risk. 
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Initial  reporting  requirements  incumbent  to  the  baseline  acquisition  process  are 
adequate.  An  acquisition  reform  such  as  the  WSARA,  adds  to  the  reporting 
requirements.  As  more  reform  initiatives  are  made,  more  reporting  is  required.  Further, 
no  unnecessary  or  outdated  reporting  requirements  are  eliminated.  These  new  and  old 
reporting  requirements  compound  on  top  of  each  other,  until  only  reporting  is  happening. 
The  original  intent  of  the  reporting  is  lost,  and  no  one  has  any  idea  why  the  reporting  is 
being  done  and  whether  or  not  reports  are  necessary  at  higher  levels  anymore. 

G.  POLICY  IMPLEMENTATION  AND  GAPS 

This  section  presents  information  demonstrating  whether  there  was  deviation 
from  the  WSARA’s  intent  and  application  within  PEO  CS&CSS  and  if  so,  at  what  level, 
and  what  is  actually  implemented  within  programs. 

1.  Tracking  Acquisition  Policy 

Acquisition  policy  and  reform  is  constantly  being  reviewed  and  updated.  This 
process  is  managed  through  policies  at  all  levels  of  the  bureaucracy.  Acquisition 
managers  within  PEO  CS&CSS  have  challenges  tracking  changes  to  policy.  Some  policy 
changes  or  updates  to  procedure  are  relatively  simple  to  print  and  make  a  cheat  sheet  to 
check  your  program  efforts  against.  However,  complex  policy  reforms  and  legislative 
changes  such  as  the  WSARA  are  more  difficult,  and  this  difficulty  is  compounded  when 
one  considers  that  the  WSARA  is  only  one  of  many  policy  documents  requiring 
adherence. 

Additionally,  program  managers  are  not  pressured  to  continually  check  their 
programs  against  policy.  Resources  such  as  the  Defense  Acquisition  Portal  (DAP)  and 
Defense  Acquisition  University  (DAU)  were  cited  as  potential  sources  to  consult  when 
searching  for  policy  documents  when  evaluating  programs.  But  given  the  amount  of 
policy  and  reforms  that  exist,  this  can  be  a  lengthy  process. 

True  policy  reviews  come  when  the  program  is  reaching  certain  milestone  or 
decision  briefings  within  the  acquisition  process.  At  these  points,  the  policy  making 
personnel  scrub  documents  and  procedures  against  policy  requirements,  which  assists 
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with  ensuring  the  program  is  in  eomplianee.  However,  due  to  the  ever  inereasing  amount 
of  ehange  and  reforms  that  takes  plaee  with  the  aequisition  proeess,  the  time  it  takes  to 
proeess  these  doeuments  and  briefings  is  ever  inereasing. 

2.  Policy  Challenges 

The  requirements  ereated  by  the  WSARA  ereated  bureaueratie  ehallenges.  One 
of  the  programs  mentioned  having  diffieulties  implementing  eompetition  throughout  the 
life  eyele  of  their  program  due  to  TDP  issues.  The  program  aeknowledges  that  full  and 
open  eompetition  is  a  requirement;  however,  if  one  eontraetor  holds  the  TDP  for  a  pieee 
of  equipment  or  system,  eompetition  is  eompromised.  Purehase  of  the  TDPs  from  the 
eontraetor  is  often  impossible  in  eireumstanees  where  the  eontraetor  put  the  TDPs  out  of 
reaeh  from  the  government  by  plaeing  sueh  a  high  eost  on  the  TDPs  that  it  is  outside  the 
program’s  funding  eapabilities. 

Poliey  reforms  and  ehanges  are  diffieult  to  implement  based  on  how  eumbersome 
the  proeess  beeomes.  When  a  poliey  is  reformed,  it  must  be  implemented.  Then 
personnel  must  be  employed  to  eheek  polieies  and  generate  a  database  for  ensuring  a 
program  is  evaluated  against  the  poliey  efforts.  Additional  personnel  are  then  deviated 
from  their  prior  duties  to  start  reporting  and  ensuring  that  the  database  and  reporting  is 
feeding  the  individuals  who  require  the  reporting.  Frustration  with  this  proeess  abounds 
as  it  shows  that  reporting  eomplianee  with  poliey  is  more  signifieant  than  aetually 
ensuring  eorreet  poliey  implementation  and  sound  program  management. 

The  major  roadbloek  programs  have  eneountered  with  implementing  the 
eompetition  poliey  from  the  WSARA  is  the  TDP  issue.  Also,  the  eumbersome  proeess  of 
traeking,  ereating  metries  to  measure  a  program  against,  reporting,  and  traeking  of 
reporting  requirements  that  are  ereated  with  eaeh  additional  ehange  to  the  aequisition 
proeess  are  disineentives  to  poliey  implementation. 

3.  Policy  Successes 

The  requirements  ereated  by  the  WSARA  have  ereated  opportunity  for  sueeess. 
Being  foreed  to  eonsider  trade-offs  between  eost,  sehedule,  and  performanee  in  the  early 
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years  of  a  program  changes  acquisition  approaches.  Successes  have  occurred  through 
utilization  of  the  DODI  5000  process  guides  and  other  instances  persist  where  individuals 
within  their  programs  utilized  various  policies  to  produce  rapid  results  when  a  combat 
need  was  identified. 

Positive  feedback  in  programs  was  not  directly  linked  to  WSARA  policy.  Most 
positive  things  that  happen  in  programs  result  from  following  acquisition  policy  and 
endeavoring  to  meet  the  needs  of  end  users.  However,  the  correlation  between  these 
positive  outcomes  and  implementation  of  a  WSARA  Title  II  policy  is  not  easily 
delineated. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A,  INTRODUCTION 

The  final  section  of  this  project  addresses  our  research  conclusions  and 
recommendations.  These  items  are  grouped  within  the  categories  of  quantitative, 
organizational  and  procedural,  Title  II  WSARA,  and  summary  conclusions  and 
recommendations.  We  initially  address  statistical  conclusions  drawn  from  the  DAMIR 
analysis.  The  remaining  conclusions  and  recommendations  are  based  primarily  on 
interview  trends  as  well  as  any  other  research  findings. 

1.  Conclusions  Summary 

There  is  no  direct,  measurable  linkage  between  Title  II  WSARA  policies  and 
improvements  to  processes,  organizational  structure,  costs,  schedule,  and 
performance.  Priorities  and  incentives  drive  the  acquisition  process.  Within  the 
acquisition  process  and  as  new  acquisition  officers,  we  are  constantly  counseled  to 
consider  cost,  schedule,  and  performance  within  acquisition  programs.  Indeed,  the 
research  indicated  that  these  three  items,  along  with  risk,  as  equally  important.  However, 
as  interviews  were  conducted,  discrepancies  amongst  how  these  items  are  prioritized 
within  a  program  surfaced. 

Managers  and  workers  of  programs  respond  to  incentives.  Incentives  present 
themselves  in  the  form  of  career  advancement,  excellent  evaluations,  praise,  and  many 
others.  When  and  why  these  incentives  are  delivered  to  a  recipient  establish  the 
program’s  priorities.  So,  while  a  manager  may  say  cost,  schedule,  performance,  and  risk 
are  equally  important,  their  actions  in  only  rewarding  when  funds  are  fully  obligated  lead 
subordinates  to  determine  that  the  true  priority  is  money  obligated  and  spent  in  a  program 
vice  what  were  the  stated  priorities. 

Adding  to  the  already  numerous  policy  documents  that  exist  to  govern  the 

acquisition  process  is  not  the  most  effective  or  efficient  manner  to  improve  acquisitions. 

Rather,  enforcing  requirements  for  PdMs  and  other  representatives  to  adhere  to  the 

acquisition  process  while  establishing  priorities  at  the  PEO  and  below  level  (priorities 
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determined  from  evolving  needs  and  fiseal  requirements)  and  incentivizing  all  involved 
to  adhere  to  these  priorities  will  deliver  desired  improvements. 

B,  CONCLUSIONS 

1,  Quantitative 

It  is  clear  that  MDAP  cost  variability  is  in  more  control  on  current  estimate  and 
PAUC  and  APUC  baseline  comparisons  since  the  implementation  of  the  WSARA. 

As  stated  in  the  data  analysis  section,  we  focused  on  analyzing  the  measures  of 
central  tendency  for  the  selected  elements  of  cost  in  the  current  estimate,  PAUC  change 
against  baseline,  and  APUC  change  against  baseline  from  each  Army  MDAP  SARs  that 
were  available  from  2005  to  2012. 

The  reader’s  focus  should  shift  to  the  coefficient  of  variation  (CV),  as  this 
measurement  equalizes  the  playing  field  because  it  is  has  no  unit,  nullifying  actual  dollar 
figures  or  program  baseline  estimates,  and  compares  the  program  standard  deviation 
divided  by  its  average,  giving  relative  magnitude  and  relative  variability.  For  example, 
from  Jeff  Sauro’s  website: 

if  the  mean  is  80  and  standard  deviation  is  12,  the  cv  =  12/80  =  .15  or 
15%.  If  the  standard  deviation  is  .20  and  the  mean  is  .50,  then  the  cv  = 

.20/.50  =  .4  or  40%.  So  knowing  nothing  else  about  the  data,  the  CV  helps 
us  see  that  even  a  lower  standard  deviation  doesn’t  mean  less  variable 
data. 

We  calculated  the  differences  of  coefficient  of  variation  (CV)  between  the 
samples  (for  each  SAR  cost  element)  before  the  WSARA  and  then  after  WSARA 
implementation. 

For  our  example,  reference  Table  1  of  this  project.  All  of  the  MDAPs’  APUC  CV 
before  the  WSARA  was  99.01%  and  after  was  81.01%,  showing  the  change  of  -18.78% 
in  Table  2  below.  For  the  programs  that  only  decreased  in  CV,  they  did  so  by  -85.34%, 
shown  in  Table  2  (with  the  data  computation  drawn  from  Table  1).  Each  of  the  values  in 
Table  2  that  decreased  indicates  a  cost  measurement  that  is  more  in  control  (and  less  in 
control  if  the  CV  increased). 
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Since  WSARA  implementation,  the  sampled  programs  have  improved  in  cost 
control.  The  spirit  of  the  law  intended  to  have  more  control  on  programs  and  lower 
eosts. 

Table  2  summarizes  the  changes  on  programs  that  inereased  or  decreased  before 
the  implementation  of  the  WSARA  and  after.  In  addition,  there  is  a  combined  net  ehange 
in  eaeh  measurement  showing  the  total  change. 


Table  2.  Summary  SAR  MDAP  Changes  In  Central  Tendency 


SAR  Report  Data _ Average _ Stdev _ CV 

Current  Estimate  $160.09  $900.64  13,64% 

PAUC%  Change  34.88%  0.1803  58.82% 

APUC%  Change  6.10%  0.0671  33.40% 

PROGRAMS  THAT  DECREASED 


SAR  Report  Data _ Average _ Stdev _ CV 

Current  Estimate  $(4,538.13)  $(970.56)  -14,24% 

PAUC%  Change  -7.45%  -0.0596  -131.12% 

APUC%  Change  -9.15%  -0.0655  -85.34% 

COMBINED  NET  CHANGE  FOR  ALL  PROGRAMS 


SAR  Report  Data 

Average 

Stdev 

CV 

Current  Estimate 

$(1,514.21) 

$(306.97) 

-2.27% 

PAUC  %  Change 

4.98% 

0.0379 

-69.63% 

APUC  %  Change 

-4.06% 

0.0125 

-18.78% 

It  is  elear  that  MDAP  cost  variability  is  in  more  eontrol  on  current  estimate  and 
PAUC  and  APUC  baseline  comparisons  since  the  implementation  of  the  WSARA. 
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2,  Organizational  and  Procedural  Impacts 

Our  research  indicated  no  true  change  to  organizational  structure,  task  allocation, 
or  administrative  functions  directly  linked  to  WSARA  implementation.  The 
organizational  structure  at  the  PEO  level  and  above,  is  very  structured.  This  is  necessary 
to  establish  clear  linkages  and  lines  of  responsibility  from  PEO  to  congressional  leaders, 
DAEs,  and  others.  However,  at  the  PdM  and  below  level,  the  organizational  structure  is 
highly  fluid  and  was  different  within  each  program  that  we  interviewed,  as  were  their 
procedures.  The  structure  was  tailored  to  focus  personnel  on  the  priorities  of  the 
program.  Some  PdMs  managed  people  and  tasks  by  functional  category  (logistics, 
operations,  administration,  engineering  etc.)  while  others  were  organized  by  product  line 
or  subcategory  within  the  program  itself. 

This  decentralized  approach  to  managing  structure  and  procedures  at  the  PdM 
level  and  below  is  effective  and  efficient  according  to  program  leaders.  While  none 
interviewed  stated  the  WSARA  as  a  reason  for  changing  organizational  structure  and 
procedures,  it  is  possible  that,  due  to  influence  from  representatives  at  the  PEO  level  and 
above  whose  actions  may  have  been  motivated  by  the  WSARA,  the  reform  is  partially 
responsible  for  some  organizational  changes. 

The  research  suggests  that  a  negative  organizational  and  procedural  impact 
resulting  from  acquisition  reforms  and  changes  to  policy.  This  negative  impact  is  seen  in 
the  increased  amount  of  reporting  and  tracking  or  reporting  that  is  required  each  time  new 
policy  is  implemented.  Increased  reporting  leads  to  allocation  of  resources  (personnel 
and  time)  to  reporting  requirements,  decreasing  resources  that  have  previously  been 
focused  on  other  important  management  tasks. 

3.  WSARA  Title  II  Policy  Application  Conclusions 

Title  II  WSARA  policies  focus  on  evaluation  of  tradeoffs  between  cost,  performance 
and  schedule,  prototyping,  maintaining  competition  throughout  a  program’s  life  cycle, 
addressing  systemic  problems,  and  terminating  product  lines.  Throughout  our  initial  research 
and  program  representative  interviews,  we  determined  that  the  acquisition  process  and 
policies  already  have  procedures  and  rules  concerning  what  Title  II  WSARA  requires. 
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Additionally,  all  programs  do  not  fall  in  the  same  spaee  and  time  within  the 
aequisition  proeess.  So,  when  a  new  requirement  to  ensure  eompetition  throughout  a 
produet’s  life  eyele  is  implemented  in  a  program  in  its  latter  (Operations  and  Support) 
phase  of  the  aequisition  proeess,  this  may  lead  to  sehedule  delays.  The  research 
indicated  that,  in  such  programs,  the  TDPs  are  often  out  of  their  reach.  New  competition 
requirements  likely  require  waivers  if  competition  cannot  take  place,  which  may  result  in 
schedule  overruns  due  to  processing  delays.  This  is  one  example  of  many  possible 
scenarios  where  policy  changes  lead  to  delays  or  other  tradeoffs  within  programs. 

C.  RECOMMENDATIONS 

The  quantitative  findings  indicate  programs  that  are  lower  and  more  in  control  on 
average  since  the  implementation  of  WSARA.  However,  a  larger  sample  size  of  cost 
data  from  SARs  or  other  sources  would  increase  statistical  significance.  Although  there 
is  not  direct  causation  identified,  there  is  a  clear  correlation  of  the  WSARA’s  impacts  on 
Army  MDAPs.  We  recommend  continuing  to  implement  WSARA  policy  requirements 
and  lessen  other  acquisition  reform  to  control  for  the  WSARA  cause  and  effect. 

To  say  that  each  program  is  the  same  and  thereby  should  maintain  a  pre¬ 
designated  organizational  structure  at  the  PdM  level  and  below  would  force  programs  to 
reorient  themselves  in  a  manner  that  is  likely  to  be  less  capable  and  effective  than  they 
are  currently.  PdMs  must  continually  evaluate  their  organizational  structure  and 
procedures  in  order  to  determine  how  to  best  tailor  their  programs  to  meet  the 
requirements  in  accordance  with  where  their  program  falls  in  the  acquisition  process  and 
priorities  established  by  their  PEO  and  above. 

PEO  and  above  representatives  should  be  responsible  for  establishing  procedural 
and  organizational  priorities  that  are  in  line  with  policies  and  reforms.  People  respond  to 
incentives  and  react  according  to  that  which  is  rewarded.  When  PEOs  establish  priorities 
and  evaluate  based  on  these  priorities  it  ensures  PdMs  focus  on  those  priorities  without 
needing  to  know  each  change  to  policy  that  occurs. 

Our  research  found  that  policy-makers  may  not  be  evaluating  whether  or  not  a 
policy  reform  is  required  prior  to  implementing  a  change.  In  other  words,  policy  changes 
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become  more  about  a  policy-maker  wanting  to  make  his  or  her  mark  on  history  while 
responding  to  pressures  from  external  forces  such  as  public  criticism  to  fix  military 
acquisitions.  Thus,  a  change  becomes  implemented  which  requires  reporting,  waivers 
and  tracking  or  all  of  these  items,  resulting  in  a  potential  detriment  to  effective  program 
management. 

Policy-makers  must  evaluate  the  baseline  acquisition  process  and  its  supporting 
policy  by  ensuring  that  reforms  are  not  made  when  they  are  not  needed.  Further,  reforms 
need  not  be  repetitive  in  nature.  When  repetitive  changes  are  made  to  policies,  this  is  an 
indicator  that  incentives  and  priorities  are  mismatched.  A  new  reform  is  not  necessary. 
Rather,  PEOs  and  PdMs  need  to  understand  what  their  priorities  are  and  be  managed 
through  incentives  to  adhere  to  those  priorities. 
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